Luma mapping- and chroma scaling-based video or image coding

ABSTRACT

According to the disclosure of the present document, for LMCS, a linear reshaper may be used, LMCS codewords (or ranges thereof) may be restricted, and in the chroma scaling of the LMCS, a single chroma residual scaling factor may be used. Accordingly, the resources/cost (of software or hardware) necessary for an LMCS procedure may be minimized, and latency in coding may be eliminated, thus enabling the LMCS procedure to be efficiently performed.

BACKGROUND OF THE DISCLOSURE Field of the Disclosure

The present document relates to luma mapping and chroma scaling basedvideo or image coding.

Related Art

Recently, demand for high-resolution, high-quality image/video such as4K or 8K or higher ultra high definition (UHD) image/video has increasedin various fields. As image/video data has high resolution and highquality, the amount of information or bits to be transmitted increasesrelative to the existing image/video data, and thus, transmitting imagedata using a medium such as an existing wired/wireless broadband line oran existing storage medium or storing image/video data using existingstorage medium increase transmission cost and storage cost.

In addition, interest and demand for immersive media such as virtualreality (VR) and artificial reality (AR) content or holograms hasrecently increased and broadcasting for image/video is havingcharacteristics different from reality images such as game images hasincreased.

Accordingly, a highly efficient image/video compression technology isrequired to effectively compress, transmit, store, and reproduceinformation of a high-resolution, high-quality image/video havingvarious characteristics as described above.

In addition, there is a discussion about techniques using luma mappingwith chroma scaling (LMCS) to improve compression efficiency andincrease subjective/objective visual quality. In order to efficientlyapply these techniques, there is a need for a method for efficientlysignaling related information.

SUMMARY

According to an embodiment of the present document, a method and anapparatus for increasing image coding efficiency are provided.

According to an embodiment of the present document, efficient filteringapplication method and apparatus are provided.

According to an embodiment of the present document, efficient LMCSapplication method and apparatus are provided.

According to an embodiment of the present document, the LMCS codewords(or a range thereof) may be constrained.

According to an embodiment of the present document, a single chromaresidual scaling factor may be used in chroma scaling in LMCS.

According to an embodiment of the present document, a linear reshapermay be used for LMCS.

According to an embodiment of the present document, a video/imagedecoding method performed by a decoding apparatus is provided.

According to an embodiment of the present document, a decoding apparatusfor performing video/image decoding is provided.

According to an embodiment of the present document, a video/imageencoding method performed by an encoding apparatus is provided.

According to an embodiment of the present document, an encodingapparatus for performing video/image encoding is provided.

According to one embodiment of the present document, there is provided acomputer-readable digital storage medium in which encoded video/imageinformation, generated according to the video/image encoding methoddisclosed in at least one of the embodiments of the present document, isstored.

According to an embodiment of the present document, there is provided acomputer-readable digital storage medium in which encoded information orencoded video/image information, causing to perform the video/imagedecoding method disclosed in at least one of the embodiments of thepresent document by the decoding apparatus, is stored.

Advantageous Effects

According to an embodiment of the present document, overall image/videocompression efficiency may be improved.

According to an embodiment of the present document, subjective/objectivevisual quality may be improved through efficient filtering.

According to an embodiment of the present document, the LMCS process forimage/video coding may be efficiently performed.

According to an embodiment of the present document, it is possible tominimize resources/costs (of software or hardware) required for the LMCSprocess.

According to an embodiment of the present document, hardwareimplementation for the LMCS process may be facilitated.

According to an embodiment of the present document, a division operationrequired for derivation of LMCS codewords in mapping (reshaping) can beremoved or minimized by constraint of the LMCS codewords (or rangethereof).

According to an embodiment of the present document, latency according topiecewise index identification may be removed by using a single chromaresidual scaling factor.

According to an embodiment of the present document, a chroma residualscaling process can be performed without depending on (reconstructionof) a luma block by using the linear reshaper, and thus latency inscaling can be removed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example of a video/image coding system to whichthe embodiments of the present document may be applied.

FIG. 2 is a diagram schematically illustrating a configuration of avideo/image encoding apparatus to which the embodiments of the presentdocument may be applied.

FIG. 3 is a diagram schematically illustrating a configuration of avideo/image decoding apparatus to which the embodiments of the presentdocument may be applied.

FIG. 4 exemplarily shows a hierarchical structure for a codedimage/video.

FIG. 5 exemplarily illustrates a hierarchical structure of a CVSaccording to an embodiment of the present document.

FIG. 6 illustrates an exemplary LMCS structure according to anembodiment of the present document.

FIG. 7 illustrates an LMCS structure according to another embodiment ofthe present document.

FIG. 8 shows a graph representing an exemplary forward mapping.

FIG. 9 is a flowchart illustrating a method for deriving a chromaresidual scaling index according to an embodiment of the presentdocument.

FIG. 10 illustrates a linear fitting of pivot points according to anembodiment of the present document.

FIG. 11 illustrates one example of linear reshaping (or linearreshaping, linear mapping) according to an embodiment of the presentdocument.

FIG. 12 and FIG. 13 schematically show an example of a video/imageencoding method and related components according to embodiment(s) of thepresent document.

FIG. 14 and FIG. 15 schematically show an example of an image/videodecoding method and related components according to an embodiment of thepresent document.

FIG. 16 shows an example of a content streaming system to whichembodiments disclosed in the present document may be applied.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

The present document may be modified in various forms, and specificembodiments thereof will be described and shown in the drawings.However, the embodiments are not intended for limiting the presentdocument. The terms used in the following description are used to merelydescribe specific embodiments, but are not intended to limit the presentdocument. An expression of a singular number includes an expression ofthe plural number, so long as it is clearly read differently. The termssuch as “include” and “have” are intended to indicate that features,numbers, steps, operations, elements, components, or combinationsthereof used in the following description exist and it should be thusunderstood that the possibility of existence or addition of one or moredifferent features, numbers, steps, operations, elements, components, orcombinations thereof is not excluded.

Meanwhile, each configuration in the drawings described in the presentdocument is shown independently for the convenience of descriptionregarding different characteristic functions, and does not mean thateach configuration is implemented as separate hardware or separatesoftware. For example, two or more components among each component maybe combined to form one component, or one component may be divided intoa plurality of components. Embodiments in which each component isintegrated and/or separated are also included in the scope of thedisclosure of the present document.

Hereinafter, examples of the present embodiment will be described indetail with reference to the accompanying drawings. In addition, likereference numerals are used to indicate like elements throughout thedrawings, and the same descriptions on the like elements will beomitted.

FIG. 1 illustrates an example of a video/image coding system to whichthe embodiments of the present document may be applied.

Referring to FIG. 1, a video/image coding system may include a firstdevice (a source device) and a second device (a reception device). Thesource device may transmit encoded video/image information or data tothe reception device through a digital storage medium or network in theform of a file or streaming.

The source device may include a video source, an encoding apparatus, anda transmitter. The receiving device may include a receiver, a decodingapparatus, and a renderer. The encoding apparatus may be called avideo/image encoding apparatus, and the decoding apparatus may be calleda video/image decoding apparatus. The transmitter may be included in theencoding apparatus. The receiver may be included in the decodingapparatus. The renderer may include a display, and the display may beconfigured as a separate device or an external component.

The video source may acquire video/image through a process of capturing,synthesizing, or generating the video/image. The video source mayinclude a video/image capture device and/or a video/image generatingdevice. The video/image capture device may include, for example, one ormore cameras, video/image archives including previously capturedvideo/images, and the like. The video/image generating device mayinclude, for example, computers, tablets and smartphones, and may(electronically) generate video/images. For example, a virtualvideo/image may be generated through a computer or the like. In thiscase, the video/image capturing process may be replaced by a process ofgenerating related data.

The encoding apparatus may encode input video/image. The encodingapparatus may perform a series of procedures such as prediction,transform, and quantization for compaction and coding efficiency. Theencoded data (encoded video/image information) may be output in the formof a bitstream.

The transmitter may transmit the encoded image/image information or dataoutput in the form of a bitstream to the receiver of the receivingdevice through a digital storage medium or a network in the form of afile or streaming. The digital storage medium may include variousstorage mediums such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and thelike. The transmitter may include an element for generating a media filethrough a predetermined file format and may include an element fortransmission through a broadcast/communication network. The receiver mayreceive/extract the bitstream and transmit the received bitstream to thedecoding apparatus.

The decoding apparatus may decode the video/image by performing a seriesof procedures such as dequantization, inverse transform, and predictioncorresponding to the operation of the encoding apparatus.

The renderer may render the decoded video/image. The renderedvideo/image may be displayed through the display.

The present document relates to video/image coding. For example, amethod/embodiment disclosed in the present document may be applied to amethod disclosed in the versatile video coding (VVC) standard, theessential video coding (EVC) standard, the AOMedia Video 1 (AV1)standard, the 2nd generation of audio video coding standard (AVS2) orthe next generation video/image coding standard (e.g., H.267, H.268, orthe like).

The present document suggests various embodiments of video/image coding,and the above embodiments may also be performed in combination with eachother unless otherwise specified.

In the present document, a video may refer to a series of images overtime. A picture generally refers to the unit representing one image at aparticular time frame, and a slice/tile refers to the unit constitutinga part of the picture in terms of coding. A slice/tile may include oneor more coding tree units (CTUs). One picture may consist of one or moreslices/tiles. One picture may consist of one or more tile groups. Onetile group may include one or more tiles. A brick may represent arectangular region of CTU rows within a tile in a picture. A tile may bepartitioned into a multiple bricks, each of which may be constructedwith one or more CTU rows within the tile. A tile that is notpartitioned into multiple bricks may also be referred to as a brick. Abrick scan may represent a specific sequential ordering of CTUspartitioning a picture, wherein the CTUs may be ordered in a CTU rasterscan within a brick, and bricks within a tile may be orderedconsecutively in a raster scan of the bricks of the tile, and tiles in apicture may be ordered consecutively in a raster scan of the tiles ofthe picture. A tile is a rectangular region of CTUs within a particulartile column and a particular tile row in a picture. The tile column is arectangular region of CTUs having a height equal to the height of thepicture and a width specified by syntax elements in the pictureparameter set. The tile row is a rectangular region of CTUs having aheight specified by syntax elements in the picture parameter set and awidth equal to the width of the picture. A tile scan is a specificsequential ordering of CTUs partitioning a picture in which the CTUs areordered consecutively in CTU raster scan in a tile whereas tiles in apicture are ordered consecutively in a raster scan of the tiles of thepicture. A slice includes an integer number of bricks of a picture thatmay be exclusively contained in a single NAL unit. A slice may consistsof either a number of complete tiles or only a consecutive sequence ofcomplete bricks of one tile. In the present document, a tile group and aslice may be used in place of each other. For example, in the presentdocument, a tile group/tile group header may be referred to as aslice/slice header.

Meanwhile, one picture may be divided into two or more subpictures. Asubpicture may be a rectangular region of one or more slices within apicture.

A pixel or a pel may mean a smallest unit constituting one picture (orimage). Also, ‘sample’ may be used as a term corresponding to a pixel. Asample may generally represent a pixel or a value of a pixel, and mayrepresent only a pixel/pixel value of a luma component or only apixel/pixel value of a chroma component.

A unit may represent a basic unit of image processing. The unit mayinclude at least one of a specific region of the picture and informationrelated to the region. One unit may include one luma block and twochroma (ex. cb, cr) blocks. The unit may be used interchangeably withterms such as block or area in some cases. In a general case, an M×Nblock may include samples (or sample arrays) or a set (or array) oftransform coefficients of M columns and N rows. Alternatively, thesample may mean a pixel value in the spatial domain, and when such apixel value is transformed to the frequency domain, it may mean atransform coefficient in the frequency domain.

In the present document, “A or B” may mean “only A”, “only B” or “both Aand B”. In other words, “A or B” in the present document may beinterpreted as “A and/or B”. For example, in the present document “A, Bor C (A, B or C)” means “only A”, “only B”, “only C”, or “anycombination of A, B and C”.

A slash (/) or comma (comma) used in the present document may mean“and/or”. For example, “A/B” may mean “A and/or B”. Accordingly, “A/B”may mean “only A”, “only B”, or “both A and B”. For example, “A, B, C”may mean “A, B, or C”.

In the present document, “at least one of A and B” may mean “only A”,“only B” or “both A and B”. Also, in the present document, theexpression “at least one of A or B” or “at least one of A and/or B” maybe interpreted the same as “at least one of A and B”.

Also, in the present document, “at least one of A, B and C” means “onlyA”, “only B”, “only C”, or “any combination of A, B and C”. Also, “atleast one of A, B or C” or “at least one of A, B and/or C” may mean “atleast one of A, B and C”.

Also, parentheses used in the present document may mean “for example”.Specifically, when “prediction (intra prediction)” is indicated, “intraprediction” may be proposed as an example of “prediction”. In otherwords, “prediction” in the present document is not limited to “intraprediction”, and “intra prediction” may be proposed as an example of“prediction”. Also, even when “prediction (i.e., intra prediction)” isindicated, “intra prediction” may be proposed as an example of“prediction”.

Technical features that are individually described in one drawing in thepresent document may be implemented individually or simultaneously.

FIG. 2 is a diagram schematically illustrating a configuration of avideo/image encoding apparatus to which the embodiments of the presentdocument may be applied. Hereinafter, what is referred to as the videoencoding apparatus may include an image encoding apparatus.

Referring to FIG. 2, the encoding apparatus 200 includes an imagepartitioner 210, a predictor 220, a residual processor 230, and anentropy encoder 240, an adder 250, a filter 260, and a memory 270. Thepredictor 220 may include an inter predictor 221 and an intra predictor222. The residual processor 230 may include a transformer 232, aquantizer 233, a dequantizer 234, and an inverse transformer 235. Theresidual processor 230 may further include a subtractor 231. The adder250 may be called a reconstructor or a reconstructed block generator.The image partitioner 210, the predictor 220, the residual processor230, the entropy encoder 240, the adder 250, and the filter 260 may beconfigured by at least one hardware component (ex. An encoder chipset orprocessor) according to an embodiment. In addition, the memory 270 mayinclude a decoded picture buffer (DPB) or may be configured by a digitalstorage medium. The hardware component may further include the memory270 as an internal/external component.

The image partitioner 210 may partition an input image (or a picture ora frame) input to the encoding apparatus 200 into one or moreprocessors. For example, the processor may be called a coding unit (CU).In this case, the coding unit may be recursively partitioned accordingto a quad-tree binary-tree ternary-tree (QTBTTT) structure from a codingtree unit (CTU) or a largest coding unit (LCU). For example, one codingunit may be partitioned into a plurality of coding units of a deeperdepth based on a quad tree structure, a binary tree structure, and/or aternary structure. In this case, for example, the quad tree structuremay be applied first and the binary tree structure and/or ternarystructure may be applied later. Alternatively, the binary tree structuremay be applied first. The coding procedure according to the presentdisclosure may be performed based on the final coding unit that is nolonger partitioned. In this case, the largest coding unit may be used asthe final coding unit based on coding efficiency according to imagecharacteristics, or if necessary, the coding unit may be recursivelypartitioned into coding units of deeper depth and a coding unit havingan optimal size may be used as the final coding unit. Here, the codingprocedure may include a procedure of prediction, transform, andreconstruction, which will be described later. As another example, theprocessor may further include a prediction unit (PU) or a transform unit(TU). In this case, the prediction unit and the transform unit may besplit or partitioned from the aforementioned final coding unit. Theprediction unit may be a unit of sample prediction, and the transformunit may be a unit for deriving a transform coefficient and/or a unitfor deriving a residual signal from the transform coefficient.

The unit may be used interchangeably with terms such as block or area insome cases. In a general case, an M×N block may represent a set ofsamples or transform coefficients composed of M columns and N rows. Asample may generally represent a pixel or a value of a pixel, mayrepresent only a pixel/pixel value of a luma component or represent onlya pixel/pixel value of a chroma component. A sample may be used as aterm corresponding to one picture (or image) for a pixel or a pel.

In the encoding apparatus 200, a prediction signal (predicted block,prediction sample array) output from the inter predictor 221 or theintra predictor 222 is subtracted from an input image signal (originalblock, original sample array) to generate a residual signal residualblock, residual sample array), and the generated residual signal istransmitted to the transformer 232. In this case, as shown, a unit forsubtracting a prediction signal (predicted block, prediction samplearray) from the input image signal (original block, original samplearray) in the encoder 200 may be called a subtractor 231. The predictormay perform prediction on a block to be processed (hereinafter, referredto as a current block) and generate a predicted block includingprediction samples for the current block. The predictor may determinewhether intra prediction or inter prediction is applied on a currentblock or CU basis. As described later in the description of eachprediction mode, the predictor may generate various information relatedto prediction, such as prediction mode information, and transmit thegenerated information to the entropy encoder 240. The information on theprediction may be encoded in the entropy encoder 240 and output in theform of a bitstream.

The intra predictor 222 may predict the current block by referring tothe samples in the current picture. The referred samples may be locatedin the neighborhood of the current block or may be located apartaccording to the prediction mode. In the intra prediction, predictionmodes may include a plurality of non-directional modes and a pluralityof directional modes. The non-directional mode may include, for example,a DC mode and a planar mode. The directional mode may include, forexample, 33 directional prediction modes or 65 directional predictionmodes according to the degree of detail of the prediction direction.However, this is merely an example, more or less directional predictionmodes may be used depending on a setting. The intra predictor 222 maydetermine the prediction mode applied to the current block by using aprediction mode applied to a neighboring block.

The inter predictor 221 may derive a predicted block for the currentblock based on a reference block (reference sample array) specified by amotion vector on a reference picture. Here, in order to reduce theamount of motion information transmitted in the inter prediction mode,the motion information may be predicted in units of blocks, sub-blocks,or samples based on correlation of motion information between theneighboring block and the current block. The motion information mayinclude a motion vector and a reference picture index. The motioninformation may further include inter prediction direction (L0prediction, L1 prediction, Bi prediction, etc.) information. In the caseof inter prediction, the neighboring block may include a spatialneighboring block present in the current picture and a temporalneighboring block present in the reference picture. The referencepicture including the reference block and the reference pictureincluding the temporal neighboring block may be the same or different.The temporal neighboring block may be called a collocated referenceblock, a co-located CU (colCU), and the like, and the reference pictureincluding the temporal neighboring block may be called a collocatedpicture (colPic). For example, the inter predictor 221 may configure amotion information candidate list based on neighboring blocks andgenerate information indicating which candidate is used to derive amotion vector and/or a reference picture index of the current block.Inter prediction may be performed based on various prediction modes. Forexample, in the case of a skip mode and a merge mode, the interpredictor 221 may use motion information of the neighboring block asmotion information of the current block. In the skip mode, unlike themerge mode, the residual signal may not be transmitted. In the case ofthe motion vector prediction (MVP) mode, the motion vector of theneighboring block may be used as a motion vector predictor and themotion vector of the current block may be indicated by signaling amotion vector difference.

The predictor 220 may generate a prediction signal based on variousprediction methods described below. For example, the predictor may notonly apply intra prediction or inter prediction to predict one block butalso simultaneously apply both intra prediction and inter prediction.This may be called combined inter and intra prediction (CIIP). Inaddition, the predictor may be based on an intra block copy (IBC)prediction mode or a palette mode for prediction of a block. The IBCprediction mode or palette mode may be used for content image/videocoding of a game or the like, for example, screen content coding (SCC).The IBC basically performs prediction in the current picture but may beperformed similarly to inter prediction in that a reference block isderived in the current picture. That is, the IBC may use at least one ofthe inter prediction techniques described in the present disclosure. Thepalette mode may be considered as an example of intra coding or intraprediction. When the palette mode is applied, a sample value within apicture may be signaled based on information on the palette table andthe palette index.

The prediction signal generated by the predictor (including the interpredictor 221 and/or the intra predictor 222) may be used to generate areconstructed signal or to generate a residual signal. The transformer232 may generate transform coefficients by applying a transformtechnique to the residual signal. For example, the transform techniquemay include at least one of a discrete cosine transform (DCT), adiscrete sine transform (DST), a karhunen-loeve transform (KLT), agraph-based transform (GBT), or a conditionally non-linear transform(CNT). Here, the GBT means transform obtained from a graph whenrelationship information between pixels is represented by the graph. TheCNT refers to transform generated based on a prediction signal generatedusing all previously reconstructed pixels. In addition, the transformprocess may be applied to square pixel blocks having the same size ormay be applied to blocks having a variable size rather than square.

The quantizer 233 may quantize the transform coefficients and transmitthem to the entropy encoder 240 and the entropy encoder 240 may encodethe quantized signal (information on the quantized transformcoefficients) and output a bitstream. The information on the quantizedtransform coefficients may be referred to as residual information. Thequantizer 233 may rearrange block type quantized transform coefficientsinto a one-dimensional vector form based on a coefficient scanning orderand generate information on the quantized transform coefficients basedon the quantized transform coefficients in the one-dimensional vectorform. Information on transform coefficients may be generated. Theentropy encoder 240 may perform various encoding methods such as, forexample, exponential Golomb, context-adaptive variable length coding(CAVLC), context-adaptive binary arithmetic coding (CABAC), and thelike. The entropy encoder 240 may encode information necessary forvideo/image reconstruction other than quantized transform coefficients(ex. values of syntax elements, etc.) together or separately. Encodedinformation (ex. encoded video/image information) may be transmitted orstored in units of NALs (network abstraction layer) in the form of abitstream. The video/image information may further include informationon various parameter sets such as an adaptation parameter set (APS), apicture parameter set (PPS), a sequence parameter set (SPS), or a videoparameter set (VPS). In addition, the video/image information mayfurther include general constraint information. In the presentdisclosure, information and/or syntax elements transmitted/signaled fromthe encoding apparatus to the decoding apparatus may be included invideo/picture information. The video/image information may be encodedthrough the above-described encoding procedure and included in thebitstream. The bitstream may be transmitted over a network or may bestored in a digital storage medium. The network may include abroadcasting network and/or a communication network, and the digitalstorage medium may include various storage media such as USB, SD, CD,DVD, Blu-ray, HDD, SSD, and the like. A transmitter (not shown)transmitting a signal output from the entropy encoder 240 and/or astorage unit (not shown) storing the signal may be included asinternal/external element of the encoding apparatus 200, andalternatively, the transmitter may be included in the entropy encoder240.

The quantized transform coefficients output from the quantizer 233 maybe used to generate a prediction signal. For example, the residualsignal (residual block or residual samples) may be reconstructed byapplying dequantization and inverse transform to the quantized transformcoefficients through the dequantizer 234 and the inverse transformer235. The adder 250 adds the reconstructed residual signal to theprediction signal output from the inter predictor 221 or the intrapredictor 222 to generate a reconstructed signal (reconstructed picture,reconstructed block, reconstructed sample array). If there is noresidual for the block to be processed, such as a case where the skipmode is applied, the predicted block may be used as the reconstructedblock. The adder 250 may be called a reconstructor or a reconstructedblock generator. The generated reconstructed signal may be used forintra prediction of a next block to be processed in the current pictureand may be used for inter prediction of a next picture through filteringas described below.

Meanwhile, luma mapping with chroma scaling (LMCS) may be applied duringpicture encoding and/or reconstruction.

The filter 260 may improve subjective/objective image quality byapplying filtering to the reconstructed signal. For example, the filter260 may generate a modified reconstructed picture by applying variousfiltering methods to the reconstructed picture and store the modifiedreconstructed picture in the memory 270, specifically, a DPB of thememory 270. The various filtering methods may include, for example,deblocking filtering, a sample adaptive offset, an adaptive loop filter,a bilateral filter, and the like. The filter 260 may generate variousinformation related to the filtering and transmit the generatedinformation to the entropy encoder 240 as described later in thedescription of each filtering method. The information related to thefiltering may be encoded by the entropy encoder 240 and output in theform of a bitstream.

The modified reconstructed picture transmitted to the memory 270 may beused as the reference picture in the inter predictor 221. When the interprediction is applied through the encoding apparatus, predictionmismatch between the encoding apparatus 200 and the decoding apparatus300 may be avoided and encoding efficiency may be improved.

The DPB of the memory 270 DPB may store the modified reconstructedpicture for use as a reference picture in the inter predictor 221. Thememory 270 may store the motion information of the block from which themotion information in the current picture is derived (or encoded) and/orthe motion information of the blocks in the picture that have alreadybeen reconstructed. The stored motion information may be transmitted tothe inter predictor 221 and used as the motion information of thespatial neighboring block or the motion information of the temporalneighboring block. The memory 270 may store reconstructed samples ofreconstructed blocks in the current picture and may transfer thereconstructed samples to the intra predictor 222.

FIG. 3 is a schematic diagram illustrating a configuration of avideo/image decoding apparatus to which the embodiment(s) of the presentdisclosure may be applied.

Referring to FIG. 3, the decoding apparatus 300 may include an entropydecoder 310, a residual processor 320, a predictor 330, an adder 340, afilter 350, a memory 360. The predictor 330 may include an interpredictor 331 and an intra predictor 332. The residual processor 320 mayinclude a dequantizer 321 and an inverse transformer 321. The entropydecoder 310, the residual processor 320, the predictor 330, the adder340, and the filter 350 may be configured by a hardware component (ex. Adecoder chipset or a processor) according to an embodiment. In addition,the memory 360 may include a decoded picture buffer (DPB) or may beconfigured by a digital storage medium. The hardware component mayfurther include the memory 360 as an internal/external component.

When a bitstream including video/image information is input, thedecoding apparatus 300 may reconstruct an image corresponding to aprocess in which the video/image information is processed in theencoding apparatus of FIG. 2. For example, the decoding apparatus 300may derive units/blocks based on block partition related informationobtained from the bitstream. The decoding apparatus 300 may performdecoding using a processor applied in the encoding apparatus. Thus, theprocessor of decoding may be a coding unit, for example, and the codingunit may be partitioned according to a quad tree structure, binary treestructure and/or ternary tree structure from the coding tree unit or thelargest coding unit. One or more transform units may be derived from thecoding unit. The reconstructed image signal decoded and output throughthe decoding apparatus 300 may be reproduced through a reproducingapparatus.

The decoding apparatus 300 may receive a signal output from the encodingapparatus of FIG. 2 in the form of a bitstream, and the received signalmay be decoded through the entropy decoder 310. For example, the entropydecoder 310 may parse the bitstream to derive information (ex.video/image information) necessary for image reconstruction (or picturereconstruction). The video/image information may further includeinformation on various parameter sets such as an adaptation parameterset (APS), a picture parameter set (PPS), a sequence parameter set(SPS), or a video parameter set (VPS). In addition, the video/imageinformation may further include general constraint information. Thedecoding apparatus may further decode picture based on the informationon the parameter set and/or the general constraint information.Signaled/received information and/or syntax elements described later inthe present disclosure may be decoded may decode the decoding procedureand obtained from the bitstream. For example, the entropy decoder 310decodes the information in the bitstream based on a coding method suchas exponential Golomb coding, CAVLC, or CABAC, and output syntaxelements required for image reconstruction and quantized values oftransform coefficients for residual. More specifically, the CABACentropy decoding method may receive a bin corresponding to each syntaxelement in the bitstream, determine a context model using a decodingtarget syntax element information, decoding information of a decodingtarget block or information of a symbol/bin decoded in a previous stage,and perform an arithmetic decoding on the bin by predicting aprobability of occurrence of a bin according to the determined contextmodel, and generate a symbol corresponding to the value of each syntaxelement. In this case, the CABAC entropy decoding method may update thecontext model by using the information of the decoded symbol/bin for acontext model of a next symbol/bin after determining the context model.The information related to the prediction among the information decodedby the entropy decoder 310 may be provided to the predictor (the interpredictor 332 and the intra predictor 331), and the residual value onwhich the entropy decoding was performed in the entropy decoder 310,that is, the quantized transform coefficients and related parameterinformation, may be input to the residual processor 320. The residualprocessor 320 may derive the residual signal (the residual block, theresidual samples, the residual sample array). In addition, informationon filtering among information decoded by the entropy decoder 310 may beprovided to the filter 350. Meanwhile, a receiver (not shown) forreceiving a signal output from the encoding apparatus may be furtherconfigured as an internal/external element of the decoding apparatus300, or the receiver may be a component of the entropy decoder 310.Meanwhile, the decoding apparatus according to the present disclosuremay be referred to as a video/image/picture decoding apparatus, and thedecoding apparatus may be classified into an information decoder(video/image/picture information decoder) and a sample decoder(video/image/picture sample decoder). The information decoder mayinclude the entropy decoder 310, and the sample decoder may include atleast one of the dequantizer 321, the inverse transformer 322, the adder340, the filter 350, the memory 360, the inter predictor 332, and theintra predictor 331.

The dequantizer 321 may dequantize the quantized transform coefficientsand output the transform coefficients. The dequantizer 321 may rearrangethe quantized transform coefficients in the form of a two-dimensionalblock form. In this case, the rearrangement may be performed based onthe coefficient scanning order performed in the encoding apparatus. Thedequantizer 321 may perform dequantization on the quantized transformcoefficients by using a quantization parameter (ex. quantization stepsize information) and obtain transform coefficients.

The inverse transformer 322 inversely transforms the transformcoefficients to obtain a residual signal (residual block, residualsample array).

The predictor may perform prediction on the current block and generate apredicted block including prediction samples for the current block. Thepredictor may determine whether intra prediction or inter prediction isapplied to the current block based on the information on the predictionoutput from the entropy decoder 310 and may determine a specificintra/inter prediction mode.

The predictor 320 may generate a prediction signal based on variousprediction methods described below. For example, the predictor may notonly apply intra prediction or inter prediction to predict one block butalso simultaneously apply intra prediction and inter prediction. Thismay be called combined inter and intra prediction (CIIP). In addition,the predictor may be based on an intra block copy (IBC) prediction modeor a palette mode for prediction of a block. The IBC prediction mode orpalette mode may be used for content image/video coding of a game or thelike, for example, screen content coding (SCC). The IBC basicallyperforms prediction in the current picture but may be performedsimilarly to inter prediction in that a reference block is derived inthe current picture. That is, the IBC may use at least one of the interprediction techniques described in the present disclosure. The palettemode may be considered as an example of intra coding or intraprediction. When the palette mode is applied, a sample value within apicture may be signaled based on information on the palette table andthe palette index.

The intra predictor 331 may predict the current block by referring tothe samples in the current picture. The referred samples may be locatedin the neighborhood of the current block or may be located apartaccording to the prediction mode. In the intra prediction, predictionmodes may include a plurality of non-directional modes and a pluralityof directional modes. The intra predictor 331 may determine theprediction mode applied to the current block by using a prediction modeapplied to a neighboring block.

The inter predictor 332 may derive a predicted block for the currentblock based on a reference block (reference sample array) specified by amotion vector on a reference picture. In this case, in order to reducethe amount of motion information transmitted in the inter predictionmode, motion information may be predicted in units of blocks,sub-blocks, or samples based on correlation of motion informationbetween the neighboring block and the current block. The motioninformation may include a motion vector and a reference picture index.The motion information may further include inter prediction direction(L0 prediction, L1 prediction, Bi prediction, etc.) information. In thecase of inter prediction, the neighboring block may include a spatialneighboring block present in the current picture and a temporalneighboring block present in the reference picture. For example, theinter predictor 332 may configure a motion information candidate listbased on neighboring blocks and derive a motion vector of the currentblock and/or a reference picture index based on the received candidateselection information. Inter prediction may be performed based onvarious prediction modes, and the information on the prediction mayinclude information indicating a mode of inter prediction for thecurrent block.

The adder 340 may generate a reconstructed signal (reconstructedpicture, reconstructed block, reconstructed sample array) by adding theobtained residual signal to the prediction signal (predicted block,predicted sample array) output from the predictor (including the interpredictor 332 and/or the intra predictor 331). If there is no residualfor the block to be processed, such as when the skip mode is applied,the predicted block may be used as the reconstructed block.

The adder 340 may be called reconstructor or a reconstructed blockgenerator. The generated reconstructed signal may be used for intraprediction of a next block to be processed in the current picture, maybe output through filtering as described below, or may be used for interprediction of a next picture.

Meanwhile, luma mapping with chroma scaling (LMCS) may be applied in thepicture decoding process.

The filter 350 may improve subjective/objective image quality byapplying filtering to the reconstructed signal. For example, the filter350 may generate a modified reconstructed picture by applying variousfiltering methods to the reconstructed picture and store the modifiedreconstructed picture in the memory 360, specifically, a DPB of thememory 360. The various filtering methods may include, for example,deblocking filtering, a sample adaptive offset, an adaptive loop filter,a bilateral filter, and the like.

The (modified) reconstructed picture stored in the DPB of the memory 360may be used as a reference picture in the inter predictor 332. Thememory 360 may store the motion information of the block from which themotion information in the current picture is derived (or decoded) and/orthe motion information of the blocks in the picture that have alreadybeen reconstructed. The stored motion information may be transmitted tothe inter predictor 260 so as to be utilized as the motion informationof the spatial neighboring block or the motion information of thetemporal neighboring block. The memory 360 may store reconstructedsamples of reconstructed blocks in the current picture and transfer thereconstructed samples to the intra predictor 331.

In the present document, the embodiments described in the filter 260,the inter predictor 221, and the intra predictor 222 of the encodingapparatus 200 may be the same as or respectively applied to correspondto the filter 350, the inter predictor 332, and the intra predictor 331of the decoding apparatus 300. The same may also apply to the unit 332and the intra predictor 331.

As described above, in video coding, prediction is performed to increasecompression efficiency. Through this, it is possible to generate apredicted block including prediction samples for a current block, whichis a block to be coded. Here, the predicted block includes predictionsamples in a spatial domain (or pixel domain) The predicted block isderived equally from the encoding device and the decoding device, andthe encoding device decodes information (residual information) on theresidual between the original block and the predicted block, not theoriginal sample value of the original block itself. By signaling to thedevice, image coding efficiency can be increased. The decoding apparatusmay derive a residual block including residual samples based on theresidual information, and generate a reconstructed block includingreconstructed samples by summing the residual block and the predictedblock, and generate a reconstructed picture including reconstructedblocks.

The residual information may be generated through transformation andquantization processes. For example, the encoding apparatus may derive aresidual block between the original block and the predicted block, andperform a transform process on residual samples (residual sample array)included in the residual block to derive transform coefficients, andthen, by performing a quantization process on the transformcoefficients, derive quantized transform coefficients to signal theresidual related information to the decoding apparatus (via abitstream). Here, the residual information may include locationinformation, a transform technique, a transform kernel, and aquantization parameter, value information of the quantized transformcoefficients etc. The decoding apparatus may performdequantization/inverse transformation process based on the residualinformation and derive residual samples (or residual blocks). Thedecoding apparatus may generate a reconstructed picture based on thepredicted block and the residual block. The encoding apparatus may alsodequantize/inverse transform the quantized transform coefficients forreference for inter prediction of a later picture to derive a residualblock, and generate a reconstructed picture based thereon.

In the present document, at least one of quantization/dequantizationand/or transform/inverse transform may be omitted. When thequantization/dequantization is omitted, the quantized transformcoefficient may be referred to as a transform coefficient. When thetransform/inverse transform is omitted, the transform coefficients maybe called coefficients or residual coefficients, or may still be calledtransform coefficients for uniformity of expression.

In the present document, a quantized transform coefficient and atransform coefficient may be referred to as a transform coefficient anda scaled transform coefficient, respectively. In this case, the residualinformation may include information on transform coefficient(s), and theinformation on the transform coefficient(s) may be signaled throughresidual coding syntax. Transform coefficients may be derived based onthe residual information (or information on the transformcoefficient(s)), and scaled transform coefficients may be derivedthrough inverse transform (scaling) on the transform coefficients.Residual samples may be derived based on an inverse transform(transform) of the scaled transform coefficients. This may beapplied/expressed in other parts of the present document as well.

Intra prediction may refer to prediction that generates predictionsamples for the current block based on reference samples in a picture towhich the current block belongs (hereinafter, referred to as a currentpicture). When intra prediction is applied to the current block,neighboring reference samples to be used for intra prediction of thecurrent block may be derived. The neighboring reference samples of thecurrent block may include samples adjacent to the left boundary of thecurrent block having a size of nW×nH and a total of 2×nH samplesneighboring the bottom-left, samples adjacent to the top boundary of thecurrent block and a total of 2×nW samples neighboring the top-right, andone sample neighboring the top-left of the current block. Alternatively,the neighboring reference samples of the current block may include aplurality of upper neighboring samples and a plurality of leftneighboring samples. In addition, the neighboring reference samples ofthe current block may include a total of nH samples adjacent to theright boundary of the current block having a size of nW×nH, a total ofnW samples adjacent to the bottom boundary of the current block, and onesample neighboring (bottom-right) neighboring bottom-right of thecurrent block.

However, some of the neighboring reference samples of the current blockmay not be decoded yet or available. In this case, the decoder mayconfigure the neighboring reference samples to use for prediction bysubstituting the samples that are not available with the availablesamples. Alternatively, neighboring reference samples to be used forprediction may be configured through interpolation of the availablesamples.

When the neighboring reference samples are derived, (i) the predictionsample may be derived based on the average or interpolation ofneighboring reference samples of the current block, and (ii) theprediction sample may be derived based on the reference sample presentin a specific (prediction) direction for the prediction sample among theperiphery reference samples of the current block. The case of (i) may becalled non-directional mode or non-angular mode and the case of (ii) maybe called directional mode or angular mode.

Furthermore, the prediction sample may also be generated throughinterpolation between the second neighboring sample and the firstneighboring sample located in a direction opposite to the predictiondirection of the intra prediction mode of the current block based on theprediction sample of the current block among the neighboring referencesamples. The above case may be referred to as linear interpolation intraprediction (LIP). In addition, chroma prediction samples may begenerated based on luma samples using a linear model. This case may becalled LM mode.

In addition, a temporary prediction sample of the current block may bederived based on filtered neighboring reference samples, and at leastone reference sample derived according to the intra prediction modeamong the existing neighboring reference samples, that is, unfilteredneighboring reference samples, and the temporary prediction sample maybe weighted-summed to derive the prediction sample of the current block.The above case may be referred to as position dependent intra prediction(PDPC).

In addition, a reference sample line having the highest predictionaccuracy among the neighboring multi-reference sample lines of thecurrent block may be selected to derive the prediction sample by usingthe reference sample located in the prediction direction on thecorresponding line, and then the reference sample line used herein maybe indicated (signaled) to the decoding apparatus, thereby performingintra-prediction encoding. The above case may be referred to asmulti-reference line (MRL) intra prediction or MRL based intraprediction.

In addition, intra prediction may be performed based on the same intraprediction mode by dividing the current block into vertical orhorizontal subpartitions, and neighboring reference samples may bederived and used in the subpartition unit. That is, in this case, theintra prediction mode for the current block is equally applied to thesubpartitions, and the intra prediction performance may be improved insome cases by deriving and using the neighboring reference samples inthe subpartition unit. Such a prediction method may be called intrasubpartitions (ISP) or ISP based intra prediction.

The above-described intra prediction methods may be called an intraprediction type separately from the intra prediction mode. The intraprediction type may be called in various terms such as an intraprediction technique or an additional intra prediction mode. Forexample, the intra prediction type (or additional intra prediction mode)may include at least one of the above-described LIP, PDPC, MRL, and ISP.A general intra prediction method except for the specific intraprediction type such as LIP, PDPC, MRL, or ISP may be called a normalintra prediction type. The normal intra prediction type may be generallyapplied when the specific intra prediction type is not applied, andprediction may be performed based on the intra prediction mode describedabove. Meanwhile, post-filtering may be performed on the predictedsample derived as needed.

Specifically, the intra prediction procedure may include an intraprediction mode/type determination step, a neighboring reference samplederivation step, and an intra prediction mode/type based predictionsample derivation step. In addition, a post-filtering step may beperformed on the predicted sample derived as needed.

When intra prediction is applied, the intra prediction mode applied tothe current block may be determined using the intra prediction mode ofthe neighboring block. For example, the decoding apparatus may selectone of most probable mode (mpm) candidates of an mpm list derived basedon the intra prediction mode of the neighboring block (ex. left and/orupper neighboring blocks) of the current block based on the received mpmindex and select one of the other remaining intro prediction modes notincluded in the mpm candidates (and planar mode) based on the remainingintra prediction mode information. The mpm list may be configured toinclude or not include a planar mode as a candidate. For example, if thempm list includes the planar mode as a candidate, the mpm list may havesix candidates. If the mpm list does not include the planar mode as acandidate, the mpm list may have three candidates. When the mpm listdoes not include the planar mode as a candidate, a not planar flag (ex.intra_luma_not_planar_flag) indicating whether an intra prediction modeof the current block is not the planar mode may be signaled. Forexample, the mpm flag may be signaled first, and the mpm index and notplanar flag may be signaled when the value of the mpm flag is 1. Inaddition, the mpm index may be signaled when the value of the not planarflag is 1. Here, the mpm list is configured not to include the planarmode as a candidate does not is to signal the not planar flag first tocheck whether it is the planar mode first because the planar mode isalways considered as mpm.

For example, whether the intra prediction mode applied to the currentblock is in mpm candidates (and planar mode) or in remaining mode may beindicated based on the mpm flag (ex. Intra_luma_mpm_flag). A value 1 ofthe mpm flag may indicate that the intra prediction mode for the currentblock is within mpm candidates (and planar mode), and a value 0 of thempm flag may indicate that the intra prediction mode for the currentblock is not in the mpm candidates (and planar mode). The value 0 of thenot planar flag (ex. Intra_luma_not_planar_flag) may indicate that theintra prediction mode for the current block is planar mode, and thevalue 1 of the not planar flag value may indicate that the intraprediction mode for the current block is not the planar mode. The mpmindex may be signaled in the form of an mpm_idx or intra_luma_mpm_idxsyntax element, and the remaining intra prediction mode information maybe signaled in the form of a rem_intra_luma_pred_mode orintra_luma_mpm_remainder syntax element. For example, the remainingintra prediction mode information may index remaining intra predictionmodes not included in the mpm candidates (and planar mode) among allintra prediction modes in order of prediction mode number to indicateone of them. The intra prediction mode may be an intra prediction modefor a luma component (sample). Hereinafter, intra prediction modeinformation may include at least one of the mpm flag (ex.Intra_luma_mpm_flag), the not planar flag (ex.Intra_luma_not_planar_flag), the mpm index (ex. mpm_idx orintra_luma_mpm_idx), and the remaining intra prediction mode information(rem_intra_luma_pred_mode or intra_luma_mpm_remainder). In the presentdocument, the MPM list may be referred to in various terms such as MPMcandidate list and candModeList. When MIP is applied to the currentblock, a separate mpm flag (ex. intra_mip_mpm_flag), an mpm index (ex.intra_mip_mpm_idx), and remaining intra prediction mode information (ex.intra_mip_mpm_remainder) for MIP may be signaled and the not planar flagis not signaled.

In other words, in general, when block splitting is performed on animage, a current block and a neighboring block to be coded have similarimage characteristics. Therefore, the current block and the neighboringblock have a high probability of having the same or similar intraprediction mode. Thus, the encoder may use the intra prediction mode ofthe neighboring block to encode the intra prediction mode of the currentblock.

For example, the encoder/decoder may configure a list of most probablemodes (MPM) for the current block. The MPM list may also be referred toas an MPM candidate list. Herein, the MPM may refer to a mode used toimprove coding efficiency in consideration of similarity between thecurrent block and neighboring block in intra prediction mode coding. Asdescribed above, the MPM list may be configured to include the planarmode or may be configured to exclude the planar mode. For example, whenthe MPM list includes the planar mode, the number of candidates in theMPM list may be 6. And, if the MPM list does not include the planarmode, the number of candidates in the MPM list may be 5.

The encoder/decoder may configure an MPM list including 5 or 6 MPMs.

In order to configure the MPM list, three types of modes can beconsidered: default intra modes, neighbor intra modes, and the derivedintra modes.

For the neighboring intra modes, two neighboring blocks, i.e., a leftneighboring block and an upper neighboring block, may be considered.

As described above, if the MPM list is configured not to include theplanar mode, the planar mode is excluded from the list, and the numberof MPM list candidates may be set to 5.

In addition, the non-directional mode (or non-angular mode) among theintra prediction modes may include a DC mode based on the average ofneighboring reference samples of the current block or a planar modebased on interpolation.

When inter prediction is applied, the predictor of the encodingapparatus/decoding apparatus may derive a prediction sample byperforming inter prediction in units of blocks. Inter prediction may bea prediction derived in a manner that is dependent on data elements (ex.sample values or motion information) of picture(s) other than thecurrent picture. When inter prediction is applied to the current block,a predicted block (prediction sample array) for the current block may bederived based on a reference block (reference sample array) specified bya motion vector on the reference picture indicated by the referencepicture index. Here, in order to reduce the amount of motion informationtransmitted in the inter prediction mode, the motion information of thecurrent block may be predicted in units of blocks, subblocks, or samplesbased on correlation of motion information between the neighboring blockand the current block. The motion information may include a motionvector and a reference picture index. The motion information may furtherinclude inter prediction type (L0 prediction, L1 prediction, Biprediction, etc.) information. In the case of inter prediction, theneighboring block may include a spatial neighboring block present in thecurrent picture and a temporal neighboring block present in thereference picture. The reference picture including the reference blockand the reference picture including the temporal neighboring block maybe the same or different. The temporal neighboring block may be called acollocated reference block, a co-located CU (colCU), and the like, andthe reference picture including the temporal neighboring block may becalled a collocated picture (colPic). For example, a motion informationcandidate list may be configured based on neighboring blocks of thecurrent block, and flag or index information indicating which candidateis selected (used) may be signaled to derive a motion vector and/or areference picture index of the current block. Inter prediction may beperformed based on various prediction modes. For example, in the case ofa skip mode and a merge mode, the motion information of the currentblock may be the same as motion information of the neighboring block. Inthe skip mode, unlike the merge mode, the residual signal may not betransmitted. In the case of the motion vector prediction (MVP) mode, themotion vector of the selected neighboring block may be used as a motionvector predictor and the motion vector of the current block may besignaled. In this case, the motion vector of the current block may bederived using the sum of the motion vector predictor and the motionvector difference.

The motion information may include L0 motion information and/or L1motion information according to an inter prediction type (L0 prediction,L1 prediction, Bi prediction, etc.). The motion vector in the L0direction may be referred to as an L0 motion vector or MVL0, and themotion vector in the L1 direction may be referred to as an L1 motionvector or MVL1. Prediction based on the L0 motion vector may be calledL0 prediction, prediction based on the L1 motion vector may be called L1prediction, and prediction based on both the L0 motion vector and the L1motion vector may be called bi-prediction. Here, the L0 motion vectormay indicate a motion vector associated with the reference picture listL0 (L0), and the L1 motion vector may indicate a motion vectorassociated with the reference picture list L1 (L1). The referencepicture list L0 may include pictures that are earlier in output orderthan the current picture as reference pictures, and the referencepicture list L1 may include pictures that are later in the output orderthan the current picture. The previous pictures may be called forward(reference) pictures, and the subsequent pictures may be called reverse(reference) pictures. The reference picture list L0 may further includepictures that are later in the output order than the current picture asreference pictures. In this case, the previous pictures may be indexedfirst in the reference picture list L0 and the subsequent pictures maybe indexed later. The reference picture list L1 may further includeprevious pictures in the output order than the current picture asreference pictures. In this case, the subsequent pictures may be indexedfirst in the reference picture list 1 and the previous pictures may beindexed later. The output order may correspond to picture order count(POC) order.

FIG. 4 exemplarily shows a hierarchical structure for a codedimage/video.

Referring to FIG. 4, coded image/video is divided into a video codinglayer (VCL) that handles the decoding process of the image/video anditself, a subsystem that transmits and stores the coded information, andNAL (network abstraction layer) in charge of function and presentbetween the VCL and the subsystem.

In the VCL, VCL data including compressed image data (slice data) isgenerated, or a parameter set including a picture parameter set (PSP), asequence parameter set (SPS), and a video parameter set (VPS) or asupplemental enhancement information (SEI) message additionally requiredfor an image decoding process may be generated.

In the NAL, a NAL unit may be generated by adding header information(NAL unit header) to a raw byte sequence payload (RBSP) generated in aVCL. In this case, the RBSP refers to slice data, parameter set, SEImessage, etc., generated in the VCL. The NAL unit header may include NALunit type information specified according to RBSP data included in thecorresponding NAL unit.

As shown in the figure, the NAL unit may be classified into a VCL NALunit and a Non-VCL NAL unit according to the RBSP generated in the VCL.The VCL NAL unit may mean a NAL unit that includes information on theimage (slice data) on the image, and the Non-VCL NAL unit may mean a NALunit that includes information (parameter set or SEI message) requiredfor decoding the image.

The above-described VCL NAL unit and Non-VCL NAL unit may be transmittedthrough a network by attaching header information according to the datastandard of the subsystem. For example, the NAL unit may be transformedinto a data format of a predetermined standard such as an H.266/VVC fileformat, a real-time transport protocol (RTP), a transport stream (TS),etc., and transmitted through various networks.

As described above, the NAL unit may be specified with the NAL unit typeaccording to the RBSP data structure included in the corresponding NALunit, and information on the NAL unit type may be stored and signaled inthe NAL unit header.

For example, the NAL unit may be classified into a VCL NAL unit type anda Non-VCL NAL unit type according to whether the NAL unit includesinformation (slice data) about an image. The VCL NAL unit type may beclassified according to the nature and type of pictures included in theVCL NAL unit, and the Non-VCL NAL unit type may be classified accordingto types of parameter sets.

The following is an example of the NAL unit type specified according tothe type of parameter set included in the Non-VCL NAL unit type.

-   -   APS (Adaptation Parameter Set) NAL unit: Type for NAL unit        including APS    -   DPS (Decoding Parameter Set) NAL unit: Type for NAL unit        including DPS    -   VPS (Video Parameter Set) NAL unit: Type for NAL unit including        VPS    -   SPS(Sequence Parameter Set) NAL unit: Type for NAL unit        including SPS    -   PPS (Picture Parameter Set) NAL unit: Type for NAL unit        including PPS    -   PH (Picture header) NAL unit: Type for NAL unit including PH

The aforementioned NAL unit types may have syntax information for theNAL unit type, and the syntax information may be stored and signaled ina NAL unit header. For example, the syntax information may benal_unit_type, and NAL unit types may be specified by a nal_unit_typevalue.

Meanwhile, as described above, one picture may include a plurality ofslices, and one slice may include a slice header and slice data. In thiscase, one picture header may be further added to a plurality of slices(a slice header and a slice data set) in one picture. The picture header(picture header syntax) may include information/parameters commonlyapplicable to the picture. In the present document, a slice may be mixedor replaced with a tile group. Also, in the present document, a sliceheader may be mixed or replaced with a tile group header.

The slice header (slice header syntax) may includeinformation/parameters that may be commonly applied to the slice. TheAPS (APS syntax) or the PPS (PPS syntax) may includeinformation/parameters that may be commonly applied to one or moreslices or pictures. The SPS (SPS syntax) may includeinformation/parameters that may be commonly applied to one or moresequences. The VPS (VPS syntax) may include information/parameters thatmay be commonly applied to multiple layers. The DPS (DPS syntax) mayinclude information/parameters that may be commonly applied to theoverall video. The DPS may include information/parameters related toconcatenation of a coded video sequence (CVS). The high level syntax(HLS) in the present document may include at least one of the APSsyntax, the PPS syntax, the SPS syntax, the VPS syntax, the DPS syntax,and the slice header syntax.

In the present document, the image/image information encoded from theencoding apparatus and signaled to the decoding apparatus in the form ofa bitstream includes not only partitioning related information in apicture, intra/inter prediction information, residual information,in-loop filtering information, etc, but also information included in aslice header, information included in the APS, information included inthe PPS, information included in an SPS, and/or information included inthe VPS.

Meanwhile, in order to compensate for a difference between an originalimage and a reconstructed image due to an error occurring in acompression coding process such as quantization, an in-loop filteringprocess may be performed on reconstructed samples or reconstructedpictures as described above. As described above, the in-loop filteringmay be performed by the filter of the encoding apparatus and the filterof the decoding apparatus, and a deblocking filter, SAO, and/or adaptiveloop filter (ALF) may be applied. For example, the ALF process may beperformed after the deblocking filtering process and/or the SAO processare completed. However, even in this case, the deblocking filteringprocess and/or the SAO process may be omitted.

Meanwhile, in order to increase coding efficiency, luma mapping withchroma scaling (LMCS) may be applied as described above. LMCS may bereferred to as a loop reshaper (reshaping). In order to increase codingefficiency, LMCS control and/or signaling of LMCS related informationmay be performed hierarchically.

FIG. 5 exemplarily illustrates a hierarchical structure of a CVSaccording to an embodiment of the present document. A coded videosequence (CVS) may include a sequence parameter set (SPS), a pictureparameter set (PPS), a tile group header, tile data, and/or CTU(s).Here, the tile group header and the tile data may be referred to as aslice header and slice data, respectively.

The SPS may include flags natively to enable tools to be used in CVS. Inaddition, the SPS may be referred to by the PPS including information onparameters that change for each picture. Each of the coded pictures mayinclude one or more coded rectangular domain tiles. The tiles may begrouped into raster scans forming tile groups. Each tile group isencapsulated with header information called a tile group header. Eachtile consists of a CTU comprising coded data. Here the data may includeoriginal sample values, prediction sample values, and its luma andchroma components (luma prediction sample values and chroma predictionsample values).

FIG. 6 illustrates an exemplary LMCS structure according to anembodiment of the present document. The LMCS structure 600 of FIG. 6includes an in-loop mapping part 610 of luma components based onadaptive piecewise linear (adaptive PWL) models and a luma-dependentchroma residual scaling part 620 for chroma components. Thedequantization and inverse transform 611, reconstruction 612, and intraprediction 613 blocks of the in-loop mapping part 610 representprocesses applied in the mapped (reshaped) domain. Loop filters 615,motion compensation or inter prediction 617 blocks of the in-loopmapping part 610, and reconstruction 622, intra prediction 623, motioncompensation or inter prediction 624, loop filters 625 block of thechroma residual scaling part 620 represent processes applied in theoriginal (non-mapped, non-reshaped) domain.

As illustrated in FIG. 6, when LMCS is enabled, at least one of theinverse mapping (reshaping) process 614, a forward mapping (reshaping)process 618, and a chroma scaling process 621 may be applied. Forexample, the inverse mapping process may be applied to a (reconstructed)luma sample (or luma samples or luma sample array) in a reconstructedpicture. The inverse mapping process may be performed based on apiecewise function (inverse) index of a luma sample. The piecewisefunction (inverse) index may identify the piece to which the luma samplebelongs. Output of the inverse mapping process is a modified(reconstructed) luma sample (or modified luma samples or modified lumasample array). The LMCS may be enabled or disabled at a level of a tilegroup (or slice), picture or higher.

The forward mapping process and/or the chroma scaling process may beapplied to generate the reconstructed picture. A picture may compriseluma samples and chroma samples. A reconstructed picture with lumasamples may be referred to as a reconstructed luma picture, and areconstructed picture with chroma samples may be referred to as areconstructed chroma picture. A combination of the reconstructed lumapicture and the reconstructed chroma picture may be referred to as areconstructed picture. The reconstructed luma picture may be generatedbased on the forward mapping process. For example, if an interprediction is applied to a current block, a forward mapping is appliedto a luma prediction sample derived based on a (reconstructed) lumasample in a reference picture. Because the (reconstructed) luma samplein the reference picture is generated based on the inverse mappingprocess, the forward mapping may be applied to the luma predictionsample thus a mapped (reshaped) luma prediction sample can be derived.The forward mapping process may be performed based on a piecewisefunction index of the luma prediction sample. The piecewise functionindex may be derived based on the value of the luma prediction sample orthe value of the luma sample in the reference picture used for interprediction. If an intra prediction (or an intra block copy (IBC)) isapplied to the current block, the forward mapping is not necessarybecause the inverse mapping process has not applied to the reconstructedsamples in the current picture yet. A (reconstructed) luma sample in thereconstructed luma picture is generated based on the mapped lumaprediction sample and a corresponding luma residual sample.

The reconstructed chroma picture may be generated based on the chromascaling process. For example, a (reconstructed) chroma sample in thereconstructed chroma picture may be derived based on a chroma predictionsample and a chroma residual sample (c_(res)) in a current block. Thechroma residual sample (c_(res)) is derived based on a (scaled) chromaresidual sample (c_(resScale)) and a chroma residual scaling factor(cScaleInv may be referred to as varScale) for the current block. Thechroma residual scaling factor may be calculated based on reshaped lumaprediction sample values for the current block. For example, the scalingfactor may be calculated based on an average luma value ave(Y′_(pred))of the reshaped luma prediction sample values Y′_(pred). For areference, in FIG. 6, the (scaled) chroma residual sample derived basedon the inverse transform/dequantization may be referred to asc_(resScale), and the chroma residual sample derived by performing the(inverse) scaling process to the (scaled) chroma residual sample may bereferred to as c_(res).

FIG. 7 illustrates an LMCS structure according to another embodiment ofthe present document. FIG. 7 is described with reference to FIG. 6.Here, the difference between the LMCS structure of FIG. 7 and the LMCSstructure 600 of FIG. 6 is mainly described. The in-loop mapping partand the luma-dependent chroma residual scaling part of FIG. 7 mayoperate the same as (similarly to) the in-loop mapping part 610 and theluma-dependent chroma residual scaling part 620 of FIG. 6.

Referring to FIG. 7, a chroma residual scaling factor may be derivedbased on luma reconstructed samples. In this case, an average luma value(avgYr) may be obtained (derived) based on the neighboring lumareconstructed samples outside the reconstructed block, not the innerluma reconstructed samples of the reconstructed block, and the chromaresidual scaling factor is derived based on the average luma value(avgYr). Here, the neighboring luma reconstructed samples may beneighboring luma reconstructed samples of the current block, or may beneighboring luma reconstructed samples of virtual pipeline data units(VPDUs) including the current block. For example, when intra predictionis applied to the target block, reconstructed samples may be derivedbased on prediction samples which are derived based on the intraprediction. In the other example, when inter prediction is applied tothe target block, the forward mapping is applied to prediction sampleswhich are derived based on the inter prediction, and reconstructedsamples are generated (derived) based on the reshaped (or forwardmapped) luma prediction samples.

The video/image information signaled through the bitstream may includeLMCS parameters (information on LMCS). LMCS parameters may be configuredas high level syntax (HLS, including slice header syntax) or the like.Detailed description and configuration of the LMCS parameters will bedescribed later. As described above, the syntax tables described in thepresent document (and the following embodiments) may beconfigured/encoded at the encoder end and signaled to the decoder endthrough a bitstream. The decoder may parse/decode information on theLMCS (in the form of syntax components) in the syntax tables. One ormore embodiments to be described below may be combined. The encoder mayencode the current picture based on the information about the LMCS andthe decoder may decode the current picture based on the informationabout the LMCS.

The in-loop mapping of luma components may adjust the dynamic range ofthe input signal by redistributing the codewords across the dynamicrange to improve compression efficiency. For luma mapping, a forwardmapping (reshaping) function (FwdMap) and an inverse mapping (reshaping)function (InvMap) corresponding to the forward mapping function (FwdMap)may be used. The FwdMap function may be signaled using a piece-wiselinear models, for example, the piece-wise linear model may have 16pieces or bins. The pieces may have the equal length. In one example,the InvMap function does not need to be signalled and is instead derivedfrom the FwdMap function. That is, the inverse mapping may be a functionof the forward mapping. For example, the inverse mapping function may bemathematically built as the symmetric function of the forward mapping asreflected by the line y=x.

An in-loop (luma) reshaping may be used to map input luma values(samples) to altered values in the reshaped domain. The reshaped valuesmay be coded and then mapped back into the original (un-mapped,un-reshaped) domain after reconstruction. To compensate for theinteraction between the luma signal and the chroma signal, chromaresidual scaling may be applied. In-loop reshaping is done by specifyinghigh level syntax for the reshaper model. The reshaper model syntax maysignal a piece-wise linear model (PWL model). For example, the reshapermodel syntax may signal a PWL model with 16 bins or pieces of equallengths. A forward lookup table (FwdLUT) and/or an inverse lookup table(InvLUT) may be derived based on the piece-wise linear model. Forexample, the PWL model pre-computes the 1024-entry forward (FwdLUT) andinverse (InvLUT) look up tables (LUT)s. As an example, when the forwardlookup table FwdLUT is derived, the inverse lookup table InvLUT may bederived based on the forward lookup table FwdLUT. The forward lookuptable FwdLUT may map the input luma values Yi to the altered values Yr,and the inverse lookup table InvLUT may map the altered values Yr to thereconstructed values Y′i. The reconstructed values Y′i may be derivedbased on the input luma values Yi.

In one example, the SPS may include the syntax of Table 1 below. Thesyntax of Table may include sps_reshaper_enabled_flag as a tool enablingflag. Here, sps_reshaper_enabled_flag may be used to specify whether thereshaper is used in a coded video sequence (CVS). That is,sps_reshaper_enabled_flag may be a flag for enabling reshaping in theSPS. In one example, the syntax of Table 1 may be a part of the SPS.

TABLE 1 Descriptor seq_parameter_set_rbsp( ) {  sps_seq_parameter_set_idue(v) ...  sps_reshaper_enabled_flag u(1)  rbsp_trailing_bits( ) }

In one example, semantics on syntax elements sps_seq_parameter_set_idand sps_reshaper_enabled_flag may be as shown in Table 2 below.

TABLE 2 sps_seq_parameter_set_id provides an identifier for the SPS forreference by other syntax elements. sps_reshaper_enabled_flag equal to 1specifies that reshaper is used in the coded video sequence (CVS).sps_reshaper_enabled_flag equal to 0 specifies that reshaper is not usedin the CVS.

In one example, the tile group header or the slice header may includethe syntax of Table 3 or Table 4 below.

TABLE 3 Descriptor tile_group_header( ) { tile_group_pic_parameter_set_id ue(v) ...  if(num_tiles_in_tile_group_minus1 > 0 ) {   offset_len_minus1 ue(v)   for(i = 0; i < num_tiles_in_tile_group_minus1; i++ )   entry_point_offset_minus1[ i ] u(v)  }  if (sps_reshaper_enabled_flag ) {   tile_group_reshaper_model_present_flagu(1)   if ( tile_group_reshaper_model_present_flag )   tile_group_reshaper_model ( )   tile_group_reshaper_enable_flag u(1)  if ( tile_group_reshaper_enable_flag && (!( qtbtt_dual_tree_intra_flag&& tile_group_type == I ) ) )   tile_group_reshaper_chroma_residual_scale_flag u(1)  } byte_alignment( ) }

TABLE 4 Descriptor slice header( ) {  slice_pic_parameter_set_id ue(v)...  if( num_tiles_in_slice_minus1 > 0 ) {   offset_len_minus1 ue(v)  for( i = 0; i < num_tiles_in_slice_minus1; i++ )   entry_point_offset_minus1[ i ] u(v)  }  if (sps_reshaper_enabled_flag ) {   slice_reshaper_model_present_flag u(1)  if ( slice_reshaper_model_present_flag )    slice_reshaper_model ( )  slice_reshaper_enable_flag u(1)   if( slice_reshaper_enable_flag &&(!( qtbtt_dual_tree_intra_flag && slice_type == I ) ) )   slice_reshaper_chroma_residual_scale_flag u(1)  }  byte_alignment( )}

Semantics of syntax elements included in the syntax of Table 3 or Table4 may include, for example, matters disclosed in the following tables.

TABLE 5 tile_group_reshaper_model_present_flag equal to 1 specifiestile_group_reshaper_model ( ) is present in tile group header.tile_group_reshaper_model_present_flag equal to 0 sp ecifiestile_group_reshaper_model( ) is not present in tile group header. Whentile_group _reshaper_model_present_flag is not present, it is inferredto be equal to 0. tile_group_reshaper_enabled_flag equal to 1 specifiesthat reshaper is enabled for the current tile group.tile_group_reshaper_enabled_flag equal to 0 specifies that reshaper isnot enabled for the current tile group. Whentile_group_reshaper_enable_flag is not pr esent, it is inferred to beequal to 0. tile_group_reshaper_chroma_residual_scale_flag equal to 1specifies that chroma resid ual scaling is enabled for the current tilegroup. tile_group_reshaper_chroma_residual_s cale_flag equal to 0specifies that chroma residual scaling is not enabled for the curre nttile group. When tile_group_reshaper_chroma_residual_scale_flag is notpresent, it is inferred to be equal to 0.

TABLE 6 slice_reshaper_model_present_flag equal to 1 specifiesslice_reshaper_model( ) is prese nt in slice header.slice_reshaper_model_present_flag equal to 0 specifies slice_reshaper_model( ) is not present in slice header. Whenslice_reshaper_model_present_flag is not present, it is inferred to beequal to 0. slice_reshaper_enabled_flag equal to 1 specifies thatreshaper is enabled for the curre nt slice. slice_reshaper_enabled_flagequal to 0 specifies that reshaper is not enabled fo r the currentslice. When slice_reshaper_enable_flag is not present, it is inferred tobe equal to 0. slice_reshaper_chroma_residual_scale_flag equal to 1specifies that chroma residual scaling is enabled for the current slice.slice_reshaper_chroma_residual_scale_flag equal to 0 specifies thatchroma residual scaling is not enabled for the current slice. Whenslice_reshaper_chroma_residual_scale_flag is not present, it is inferredto be equal to 0.

As one example, once the flag enabling the reshaping (i.e.,sps_reshaper_enabled_flag) is parsed in the SPS, the tile group headermay parse additional data (i.e., information included in Table 5 or 6above) which is used to construct lookup tables (FwdLUT and/or InvLUT).In order to do this, the status of the SPS reshaper flag(sps_reshaper_enabled_flag) may be first checked in the slice header orthe tile group header. When sps_reshaper_enabled_flag is true (or 1), anadditional flag, i.e., tile_group_reshaper_model_present_flag (orslice_reshaper_model_present_flag) may be parsed. The purpose of thetile_group_reshaper_model_present_flag (orslice_reshaper_model_present_flag) may be to indicate the presence ofthe reshaping model. For example, whentile_group_reshaper_model_present_flag (orslice_reshaper_model_present_flag) is true (or 1), it may be indicatedthat the reshaper is present for the current tile group (or currentslice). When tile_group_reshaper_model_present_flag (orslice_reshaper_model_present_flag) is false (or 0), it may be indicatedthat the reshaper is not present for the current tile group (or currentslice).

If the reshaper is present and the reshaper is enabled in the currenttile group (or current slice), the reshaper model (i.e.,tile_group_reshaper_model( ) or slice_reshaper_model( ) may beprocessed. Further to this, an additional flag,tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) may alsobe parsed. The tile_group_reshaper_enable_flag (orslice_reshaper_enable_flag) may indicate whether the reshaping model isused for the current tile group (or slice). For example, iftile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) is 0 (orfalse), it may be indicated that the reshaping model is not used for thecurrent tile group (or the current slice). Iftile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) is 1 (ortrue), it may be indicated that the reshaping model is used for thecurrent tile group (or slice).

As one example, tile_group_reshaper_model_present_flag (orslice_reshaper_model_present_flag) may be true (or 1) andtile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) may befalse (or 0). This means that the reshaping model is present but notused in the current tile group (or slice). In this case, the reshapingmodel can be used in the future tile groups (or slices). As anotherexample, tile_group_reshaper_enable_flag may be true (or 1) andtile_group_reshaper_model_present_flag may be false (or 0). In such acase, the decoder uses the reshaper from the previous initialization.

When the reshaping model (i.e., tile_group_reshaper_model( ) orslice_reshaper_model( ) and tile_group_reshaper_enable_flag (orslice_reshaper_enable_flag) are parsed, it may be determined (evaluated)whether conditions necessary for chroma scaling are present. The aboveconditions includes a condition 1 (the current tile group/slice has notbeen intra-coded) and/or a condition 2 (the current tile group/slice hasnot been partitioned into two separate coding quad tree structures forluma and chroma, i.e. the block structure for The current tilegroup/slice is not a dual tree structure). If the condition 1 and/or thecondition 2 are true and/or tile_group_reshaper_enable_flag (orslice_reshaper_enable_flag) is true (or 1), thentile_group_reshaper_chroma_residual_scale_flag (orslice_reshaper_chroma_residual_scale_flag) may be parsed. Whentile_group_reshaper_chroma_residual_scale_flag (orslice_reshaper_chroma_residual_scale_flag) is enabled (if 1 or true), itmay be indicated that chroma residual scaling is enabled for the currenttile group (or slice). Whentile_group_reshaper_chroma_residual_scale_flag (orslice_reshaper_chroma_residual_scale_flag) is disabled (if 0 or false),it may be indicated that chroma residual scaling is disabled for thecurrent tile group (or slice).

The purpose of the tile group reshaping model is to parse the data thatwould be necessary to construct the lookup tables (LUTs). These LUTs areconstructed on the idea that the distribution of an allowable range ofluma values can be divided into a plurality of bins (ex. 16 bins) whichcan be represented using a set of 16 PWL system of equations. Therefore,any luma value that lies within a given bin can be mapped to an alteredluma value.

FIG. 8 shows a graph representing an exemplary forward mapping. In FIG.8, five bins are illustrated exemplarily.

Referring to FIG. 8, the x-axis represents input luma values, and they-axis represents altered output luma values. The x-axis is divided into5 bins or slices, each bin of length L. That is, the five bins mapped tothe altered luma values have the same length. The forward lookup table(FwdLUT) may be constructed using data (i.e., reshaper data) availablefrom the tile group header, and thus mapping may be facilitated.

In one embodiment, output pivot points associated with the bin indicesmay be calculated. The output pivot points may set (mark) the minimumand maximum boundaries of the output range of the luma codewordreshaping. The calculation process of the output pivot points may beperformed by computing a piecewise cumulative distribution function(CDF) of the number of codewords. The output pivot range may be slicedbased on the maximum number of bins to be used and the size of thelookup table (FwdLUT or InvLUT). As one example, the output pivot rangemay be sliced based on a product between the maximum number of bins andthe size of the lookup table (size of LUT*maximum number of binindices). For example, if the product between the maximum number of binsand the size of the lookup table is 1024, the output pivot range may besliced into 1024 entries. This serration of the output pivot range maybe performed (applied or achieved) based on (using) a scaling factor. Inone example, the scaling factor may be derived based on Equation 1below.

SF=(y2−y1)*(1<<FP_PREC)+c  [Equation 1]

In Equation 1, SF denotes a scaling factor, and y1 and y2 denote outputpivot points corresponding to each bin. Also, FP_PREC and c may bepredetermined constants. The scaling factor determined based on Equation1 may be referred to as a scaling factor for forward reshaping.

In another embodiment, with respect to inverse reshaping (inversemapping), for a defined range of the bins to be used (i.e., fromreshaper_model_min_bin_idx to reshape_model_max_bin_idx), the inputreshaped pivot points which correspond to the mapped pivot points of theforward LUT and the mapped inverse output pivot points (given by binindex under consideration*number of initial codewords) are fetched. Inanother example, the scaling factor SF may be derived based on Equation2 below.

SF=(y2−y1)*(1<<FP_PREC)/(x2−x1  [Equation 2]

In Equation 2, SF denotes a scaling factor, x1 and x2 denote input pivotpoints, and y1 and y2 denote output pivot points corresponding to eachpiece (bin) (output pivot points of the inverse mapping). Here, theinput pivot points may be pivot points mapped based on a forward lookuptable (FwdLUT), and the output pivot points may be pivot pointsinverse-mapped based on an inverse lookup table (InvLUT). Also, FP_PRECmay be a predetermined constant value. FP_PREC of Equation 2 may be thesame as or different from FP_PREC of Equation 1. The scaling factordetermined based on Equation 2 may be referred to as a scaling factorfor inverse reshaping. During inverse reshaping, partitioning of inputpivot points may be performed based on the scaling factor of Equation 2.The scaling factor SF is used to slice the range of input pivot points.Based on the partitioned input pivot points, bin indices in the rangefrom 0 to the minimum bin index (reshaper_model_min_bin_idx) and/or fromthe minimum bin index (reshaper_model_min_bin_idx) to the maximum binindex (reshape_model_max_bin_idx) are assigned the pivot values thatcorrespond to the minimum and maximum bin values.

Table 7 below shows the syntax of the reshaper model according to anembodiment. The reshaper model may be referred to as a reshaping modelor an LMCS model. Here, the reshaper model has been exemplarilydescribed as a tile group reshaper, but the present document is notnecessarily limited by this embodiment. For example, the reshaper modelmay be included in the APS, or the tile group reshaper model may bereferred to as a slice reshaper model or LMCS data. Also, the prefixreshaper_model or Rsp may be used interchangeably with lmcs. Forexample, in the following tables and the description below,reshaper_model_min_bin_idx, reshaper_model_delta_max_bin_idx,reshaper_model_max_bin_idx, RspCW, RsepDeltaCW may be usedinterchangeably with lmcs_min_bin_idx, lmcs_delta_cs_bin_jdx, lmx,lmcs_delta_csDcselta_idx, lmx, respectively.

TABLE 7 Descriptor tile_group_reshaper_model ( ) { reshaper_model_min_bin_idx ue(v)  reshaper_model_delta_max_bin_idxue(v)  reshaper_model_bin_delta_abs_cw_prec_minus1 ue(v)  for ( i =reshaper_model_min_bin_idx; i <= reshaper_model_max_bin_idx, i++ ) {  reshape_model_bin_delta_abs_CW [ i ] u(v)   if (reshaper_model_bin_delta_abs_CW[ i ] ) > 0 )   reshaper_model_bin_delta_sign_CW_flag[ i ] u(1)  } }

The semantics of syntax elements included in the syntax of Table 7 mayinclude, for example, matters disclosed in the following table.

TABLE 8 reshape_model_min_bin_idx specifies the minimum bin (or piece)index to be used in the reshaper construction process. The value ofreshape_model_min_bin_idx shall be in the range of 0 to MaxBinIdx,inclusive. The value of MaxBinIdx shall be equal to 15.reshape_model_delta_max_bin_idx specifies the maximum allowed bin (orpiece) index MaxBmIdx minus the maximum bin index to be used in thereshaper construction process. The value of reshape_model_max_bin_idx isset equal to MaxBinIdx − reshape_model_delta_max_bin_idx.reshaper_model_bin_delta_abs_cw_prec_minus1 plus 1 specifies the numberof bits used for the representation of the syntaxreshape_model_bin_delta_abs_CW[ i ]. reshape_model_bin_delta_abs_CW[ i ]specifies the absolute delta codeword value for the ith bin.reshaper_model_bin_delta_sign_CW_flag[ i ] specifies the sign ofreshape_model_bin_delta_abs_CW[ i ] as follows: - Ifreshape_model_bin_delta_sign_CW_flag[ i ] is equal to 0, thecorresponding variable RspDeltaCW[ i ] is a positive value. - Otherwise( reshape_model_bin_delta_sign_CW_flag[ i ] is not equal to 0 ), thecorresponding variable RspDeltaCW[ i ] is a negative value. The variableOrgCW is derived as follows: OrgCW = (1 << BitDepthY ) / 16 Whenreshape_model_bin_delta_sign_CW_flag[ i ] is not present, it is inferredto be equal to 0. The variable RspDeltaCW[ i ] = (1 −2*reshape_model_bin_delta_sign_CW [ i ]) *reshape_model_bin_delta_abs_CW [ i ]; The variable RspCW[ i ] is derivedas following steps: The variable OrgCW is set equal to (1 <<BitDepth_(Y)) / ( MaxBinIdx + 1). - If reshaper_model_min_bin_idx < = i<= reshaper_model_max_bin_idx RspCW[ i ] = OrgCW + RspDeltaCW[ i ]. -Otherwise, RspCW[ i ] = 0. The value of RspCW[ i ] shall be in the rangeof (OrgCW>>3) to (OrgCW<<3 − 1), inclusive. The variables InputPivot[ i] with i in the range of 0 to MaxBinIdx + 1, inclusive are derived asfollows: InputPivot[ i ] = i * OrgCW  The variable ReshapePivot[ i ]with i in the range of 0 to MaxBinIdx + 1, inclusive, the  variableScaleCoef[ i ] and InvScaleCoeff[ i ]with i in the range of 0 toMaxBinIdx , inclusive,  are derived as follows:   shiftY = 11  ReshapePivot[ 0 ] = 0;   for( i = 0; i <= MaxBinIdx ; i++) {   ReshapePivot[ i + 1 ] = ReshapePivot[ i ] + RspCW[ i ]     ScaleCoef[i ] = ( RspCW[ i ] * (1 << shiftY) + (1 << (Log2(OrgCW) − 1))) >>  (Log2(OrgCW))     if ( RspCW[ i ] == 0 )     InvScaleCoeff[ i ] = 0    else     InvScaleCoeff[ i ] = OrgCW * (1 << shiftY) / RspCW[ i ]   } The variable ChromaScaleCoef[ i ] with i in the range of 0 to MaxBinIdx, inclusive, are  derived as follows:  if ( lmcsCW[ i ] = = 0 )   ChromaScaleCoeff[ i ] = (1 << 11)  else    ChromaScaleCoeff[ i ] =InvScaleCoeff[ i ]

The inverse mapping process for the luma sample according to the presentdocument may be described in a form of the standard document as shown inthe table below.

TABLE 9 Inverse mapping process for a luma sample Input to this processis a luma sample lumaSample. Output of this process is a modified lumasample invLumaSample. The value of invLumaSample is derived asfollows: - If slice_lmcs_enabled_flag of the slice that contains theluma sample lumaSample is equal to 1, the following ordered steps apply: 1.   The variable idxYInv is derived by invoking the identification ofpiece-wise function  index process for a luma sample as specified inclause 8.8.2.3 with lumaSample as the input   and idxYInv as the output. 2.   The variable invSample is derived as follows:  invSample =InputPivot[ idxYInv ] + ( InvScaleCoeff[ idxYInv ] *   ( lumaSample −LmcsPivot[ idxYInv ] ) + ( 1 << 10 ) ) >> 11  3.   The inverse mappedluma sample invLumaSample is derived as follows:  invLumaSample =Clip1Y( invSample ) - Otherwise, invLumaSample is set equal tolumaSample.

Identification of a piecewise function index process for a luma sampleaccording to the present document may be described in a form of thestandard document as shown in the table below. In Table 10, idxYInv maybe referred to as an inverse mapping index, and the inverse mappingindex may be derived based on reconstructed luma samples (lumaSample).

TABLE 10 Identification of piecewise function index process for a lumasample Input to this process is a luma sample lumaSample. Output of thisprocess is an index idxYInv identifing the piece to which the lumasample lumaSample belongs. The variable idxYInv is derived as follows:if ( lumaSample < LmcsPivot[ lmcs_min_bin_idx + 1 ] )  idxYInv =lmcs_min_bin_idx else if ( lumaSample >= LmcsPivot[ LmcsMaxBinIdx ] ) idxYInv = LmcsMaxBinIdx else {  for( idxYInv = lmcs_min_bin_idx;idxYInv < LmcsMaxBinIdx;  idxYInv++ ) {   if( lumaSample < LmcsPivot [idxYInv + 1 ] )    break  } }

Luma mapping may be performed based on the above-described embodimentsand examples, and the above-described syntax and components includedtherein may be merely exemplary representations, and embodiments in thepresent document are not limited by the above-mentioned tables orequations. Hereinafter, a method for performing chroma residual scaling(scaling for chroma components of residual samples) based on lumamapping is described.

The (luma-dependent) Chroma residual scaling is designed to compensatefor the interaction between the luma signal and its corresponding chromasignals. For example, whether chroma residual scaling is enabled or notis also signalled at the tile group level. In one example, if lumamapping is enabled and if dual tree partition (also known as separatechroma tree) is not applied to the current tile group, an additionalflag is signalled to indicate if the luma-dependent chroma residualscaling is enabled or not. In other example, when luma mapping is notused, or when dual tree partition is used in the current tile group,luma-dependent chroma residual scaling is disabled. In another example,the luma-dependent chroma residual scaling is always disabled for thechroma blocks whose area is less than or equal to 4.

The chroma residual scaling may be based on an average value of acorresponding luma prediction block (a luma component of a predictionblock to which an intra prediction mode and/or an inter prediction modeis applied). Scaling operations at the encoder end and/or the decoderside may be implemented with fixed-point integer arithmetic based onEquation 3 below.

c′=sign(c)*((abs(c)*s+2CSCALE_FP_PREC-1)>>CSCALE_FP_PREC)  [Equation 3]

In Equation 3, c denotes a chroma residue (chroma residual sample,chroma component of residual sample), c′ denotes a scaled chromaresidual sample (scaled chroma component of a residual sample), sdenotes a chroma residual scaling factor, and CSCALE_FP_PREC denotes a(predefined) constant value to specify precision.

FIG. 9 is a flowchart illustrating a method for deriving a chromaresidual scaling index according to an embodiment of the presentdocument. The method in FIG. 9 may be performed based on FIG. 6, andtables, equations, variables, arrays, and functions included in thedescription related to FIG. 6.

In the step S910, it may be determined whether the prediction mode forthe current block is the intra prediction mode or the inter predictionmode based on the prediction mode information. If the prediction mode isthe intra prediction mode, the current block or prediction samples ofthe current block are considered to be already in the reshaped (mapped)region. If the prediction mode is the inter prediction mode, the currentblock or the prediction samples of the current block are considered tobe in the original (unmapped, non-reshaped) region.

In the step S920, when the prediction mode is the intra prediction mode,an average of the current block (or luma prediction samples of thecurrent block) may be calculated (derived). That is, the average of thecurrent block in the already reshaped area is calculated directly. Theaverage may also be referred to as an average value, a mean, or a meanvalue.

In the step S921, when the prediction mode is the inter prediction mode,forward reshaping (forward mapping) may be performed (applied) on theluma prediction samples of the current block. Through forward reshaping,luma prediction samples based on the inter prediction mode may be mappedfrom the original region to the reshaped region. In one example, forwardreshaping of the luma prediction samples may be performed based on thereshaping model described with Table 7 above.

In the step S922, an average of the forward reshaped (forward mapped)luma prediction samples may be calculated (derived). That is, anaveraging process for the forward reshaped result may be performed.

In the step S930, a chroma residual scaling index may be calculated.When the prediction mode is the intra prediction mode, the chromaresidual scaling index may be calculated based on the average of theluma prediction samples. When the prediction mode is the interprediction mode, the chroma residual scaling index may be calculatedbased on an average of forward reshaped luma prediction samples.

In an embodiment, the chroma residual scaling index may be calculatedbased on a for loop syntax. The table below shows an exemplary for loopsyntax for deriving (calculating) the chroma residual scaling index.

TABLE 11 for( idxS = 0, idxFound = 0; idxS <= MaxBinIdx; idxS++ ) {  if((S < ReshapePivot[ idxS + 1 ] )  {   idxFound = 1   break;  } }

In Table 11, idxS represents the chroma residual scaling index, idxFoundrepresents an index identifying whether the chroma residual scalingindex satisfying the condition of the if statement is obtained, Srepresents a predetermined constant value, and MaxBinIdx represents themaximum allowable bin index. ReshapPivot[idxS+1] may be derived based onTables 7 and/or 8 described above.

In an embodiment, the chroma residual scaling factor may be derivedbased on the chroma residual scaling index. Equation 4 is an example forderiving the chroma residual scaling factor.

s=ChromaScaleCoef[idxS]  [Equation 4]

In Equation 4, s represents the chroma residual scaling factor, andChromaScaleCoef may be a variable (or array) derived based on Tables 7and/or 8 described above.

As described above, the average luma value of the reference samples maybe obtained, and the chroma residual scaling factor may be derived basedon the average luma value. As described above, the chroma componentresidual sample may be scaled based on the chroma residual scalingfactor, and the chroma component reconstruction sample may be generatedbased on the scaled chroma component residual sample.

In one embodiment of the present document, a signaling structure forefficiently applying the above-described LMCS is proposed. According tothis embodiment of the present document, for example, LMCS data may beincluded in HLS (i.e., an APS), and through the header information(i.e., picture header, slice header) that is a lower level of the APS,an LMCS model (reshaper model) may be adaptively derived by signalingthe ID of the APS, which is referred to the header information. The LMCSmodel may be derived based on LMCS parameters. Also, for example, aplurality of APS IDs may be signaled through the header information, andthrough this, different LMCS models may be applied in units of blockswithin the same picture/slice.

In one embodiment according to the present document, a method forefficiently performing an operation required for LMCS is proposed.According to the semantics described above in Table 8, a divisionoperation by the piece length lmcsCW[i] (also noted as RspCW[i] in thepresent document) is required to derive InvScaleCoeff[i]. The piecelength of the inverse mapping may not be power of 2, that means thedivision cannot be performed by bit shifting.

For example, calculating InvScaleCoeff may require up to 16 divisionsper slice. According to Table 8 described above, for 10 bit coding, therange of lmcsCW[i] is from 8 to 511, so to implement the divisionoperation by lmcsCW[i] using the LUT, the size of the LUT must be 504.Also, for 12 bit coding, the range of lmcsCW[i] is from 32 to 2047, sothe LUT size needs be 2016 to implement the division operation bylmcsCW[i] using the LUT. That is, division is expensive in hardwareimplementation, therefore it is desirable to avoid division if possible.

In one aspect of this embodiment, lmcsCW[i] may be constrained to bemultiple of a fixed number (or a predetermined number or apre-determined number). Accordingly, a lookup table (LUT) (capacity orsize of the LUT) for the division may be reduced. For example, iflmcsCW[i] becomes multiple of 2, the size of the LUT to replace divisionprocess may be reduced by half.

In another aspect of this embodiment, it is proposed that for codingwith higher internal bit depth coding, on the top of existingconstraints “The value of lmcsCW[i] shall be in the range of (OrgCW>>3)to (OrgCW<<3−1)”, further constrain lmcsCW[i] to be multiple of1<<(BitDepthY−10) if coding bit depth is higher than 10. Here, BitDepthYmay be the luma bit depth. Accordingly, the possible number of lmcsCW[i]would not vary with the coding bitdepth, and the size of LUT needed tocalculate the InvScaleCoeff do not increase for higher coding bitdepth.For example, for 12-bit internal coding bitdepth, limit the values oflmcsCW[i] being multiple of 4, then the LUT to replace division processwill be the same as what is to be used for 10-bit coding. This aspectcan be implemented alone, but it can also be implemented in combinationwith the above-mentioned aspect.

In another aspect of this embodiment, lmcsCW[i] may be constrained to anarrower range. For example, lmcsCW[i] may be constrained within therange from (OrgCW>>1) to (OrgCW<<1)−1. Then for 10 bit coding, the rangeof lmcsCW[i] may be [32, 127], and therefore it only needs a LUT havinga size of 96 to calculate InvScaleCoeff.

In another aspect of the present embodiment, lmcsCW[i] may beapproximated to closest numbers being power of 2, and used that in thereshaper design. Accordingly, the division in the inverse mapping can beperformed (and replaced) by bit shifting.

In one embodiment according to the present document, constraint of theLMCS codeword range is proposed. According to Table 8 described above,the values of the LMCS codewords are in the range from (OrgCW>>3) to(OrgCW<<3)−1. This codeword range is too wide. It may result in visualartifact issue when there are large differences between RspCW [i] andOrgCW.

According to one embodiment according to the present document, it isproposed to constrain the codeword of the LMCS PWL mapping to a narrowrange. For example, the range of lmcsCW[i] may be in the range(OrgCW>>1) to (OrgCW<<1)−1.

In one embodiment according to the present document, use of a singlechroma residual scaling factor is proposed for chroma residual scalingin LMCS. The existing method for deriving the chroma residual scalingfactor uses the average value of the corresponding luma block andderives the slope of each piece of the inverse luma mapping as thecorresponding scaling factor. In addition, the process to identify thepiecewise index requires the availability of the corresponding lumablock, it results in latency problem. This is not desirable for hardwareimplementation. According to this embodiment of the present document,scaling in a chroma block may not depend on a luma block value, and itmay not be necessary to identify a piecewise index. Therefore, thechroma residual scaling process in the LMCS can be performed without alatency issue.

In the first aspect for this embodiment, the single chroma scalingfactor may be derived from both the encoder and the decoder based on theluma LMCS information. When the LMCS luma model is received, the chromaresidual scaling factor may be updated.

The table below shows an example for obtaining the single chroma scalingfactor according to the first aspect of this embodiment.

TABLE 12 Sum = 0; for( i = lmcs_min_bin_idx ; i <= lmcs_max_bin_idx ;i++ ) {  sum += InvScaleCoeff[ i ] } ChromaScaleCoeff= sum / (lmcs_max_bin_idx− lmcs_min_bin_idx +1);

Referring to Table 12, a single chroma scaling factor (ex.ChromaScaleCoeff) may be obtained by averaging the inverse luma mappingslopes of all pieces within the lmcs_min_bin_idx and lmcs_max_bin_idx.

The table below shows another example for obtaining a single chromascaling factor according to the first aspect of this embodiment.

TABLE 13 Sum = 0; for( i = lmcs_min_bin_idx ; i <= lmcs_max_bin_idx ;i++ ) {  sum += lmcsCW[ i ] } Chroma ScaleCoeff= OrgCW *(lmcs_max_bin_idx−   lmcs_min_bin_idx +1) * (1 << 11) / sum;

Referring to Table 13, the single chroma scaling factor (ex.ChromaScaleCoeff) may be obtained from two pivot points (i.e.,lmcs_min_bin_idx, lmcs_max_bin_idx). For example, the inverse slope oflinear mapping may be used as the chroma scaling factor.

FIG. 10 illustrates a linear fitting of pivot points according to anembodiment of the present document. FIG. 10 may explain another exampleaccording to the first aspect of the embodiment.

Referring to FIG. 10, in another example of this embodiment, the singlechroma scaling factor may be obtained by linear fitting of pivot pointsto minimize an error (or mean square error) between the linear fittingand the existing PWL mapping. This example may be more accurate thansimply connecting the two pivot points at lmcs_min_bin_idx andlmcs_max_bin_idx. There are many ways to find the optimal linear mappingand such example is described below.

In one example, parameters b1 and b0 of a linear fitting equationy=b1*x+b0 for minimizing the sum of least square error may be calculatedbased on Equations 5 and/or 6 below.

$\begin{matrix}{{{b1} = \frac{\sum_{1}^{n}{\left( {x_{i} - \overset{¯}{x}} \right)\left( {y_{i} - \overset{¯}{y}} \right)}}{\sum_{1}^{n}\left( {x_{i} - \overset{¯}{x}} \right)^{2}}}} & \left\lbrack {{Equation}5} \right\rbrack\end{matrix}$ $\begin{matrix}{{b0} = {\overset{¯}{y} - {b_{1}\overset{\_}{x}}}} & \left\lbrack {{Equation}6} \right\rbrack\end{matrix}$

In Equation 7 and 8, X and y are the mean of x and y, and x, and y,represent values of the i-th pivot points. In this case, x is theoriginal luma values, and y is the reshaped luma values.

Referring to FIG. 10, another simple approximation to identify thelinear mapping is given as:

-   -   Get the linear line1 between lmcs_min_bin_idx and        lmcs_max_bin_idx    -   Sum up the differences between the pivot points by using the        linear line1 and pivot points.    -   Get the average difference avgDiff.    -   Adjust the last pivot point of the linear line according to the        average difference, e.g., 2*avgDiff    -   Use the inverse slope of the adjusted linear line as the chroma        residual scale.

According to the above-described linear fitting, the chroma scalingfactor (i.e., the inverse slope of forward mapping) may be derived(obtained) based on Equation 7 or 8 below.

InvScaleCoeff=OrgCW*(1<<11)/lmcs_pivots_linear[lmcs_min_bin_idx+1]  [Equation7]

InvScaleCoeff=OrgCW*(lmcs_max_bin_idx_−lmcs_max_bin_idx+1)*(1<<11)/lmcs_pivots_linear[lmcs_max_bin_idx+1]  [Equation8]

In the above-described equations, lmcs_pivots_linear[i] may be mappedvalues of linear mapping. With linear mapping, all pieces of PWL mappingbetween minimum and maximum bin indices may have the same LMCS codeword(lmcsCW). That is, lmcs_pivots_linear[lmcs_min_bin_idx+1] may be thesame as lmcsCW[lmcs_min_bin_idx].

In the second aspect for this embodiment, the encoder may determineparameters related to the single chroma scaling factor and signal theparameters to the decoder. In order to derive the chroma scaling factorin the decoder, the x value and the y value may be needed. Here, thevalue of x is piece lengths, for example it may be a known number (tothe decoder) and may not need to be signaled. Thus, only the y value mayneed to be signaled for chroma scaling factor derivation. For example,the y value may be a value mapped to arbitrary pivot points of a linearmapping.

The table below shows examples of syntaxes for signaling the y value forchroma scaling factor derivation.

TABLE 14 Descriptor lmcs_data ( ) {   lmcs_min_bin_idx ue(v)  lmcs_delta_max_bin_idx ue(v)   lmcs_delta_cw_prec_minus1 ue(v)   for (i = lmcs_min_bin_idx; i <= LmcsMaxBinIdx;   i++ ) {   lmcs_delta_abs_cw[i ] u(v)   if ( lmcs_delta_abs_cw[ i ] ) > 0 )   lmcs_delta_sign_cw_flag[ i ] u(1)   } if(slice_chroma_residual_scale_flag) {   lmcs_delta_abs_cw_linear u(v)  if ( lmcs_delta_abs_cw_linear ) > 0 )   lmcs_delta_sign_cw_linear_flag u(1)  } }

TABLE 15 Descriptor lmcs_data ( ) {   lmcs_min_bin_idx ue(v)  lmcs_delta_max_bin_idx ue(v)   lmcs_delta_cw_prec_minus1 ue(v)   for (i = lmcs_min_bin_idx; i <= LmcsMaxBinIdx;   i++ ) {   lmcs_delta_abs_cw[i ] u(v)   if ( lmcs_delta_abs_cw[ i ] ) > 0 )   lmcs_delta_sign_cw_flag[ i ] u(1)   }  if(slice_chroma_residual_scale_flag) {   lmcs_cw_linear u(v)  } }

One of the syntaxes of Tables 14 and 15 described above may be used tosignal the y value at any linear pivot points specified by the encoderand decoder. That is, the encoder and the decoder may derive the y valueusing the same syntax.

Referring to FIG. 10, P0, P2 include mapped values. lmcs_cw_linear inthe above tables may denote a mapped value at P0 or P1.

In one example, if lmcs_cw_linear denotes a value mapped to P0 (i.e.,lmcs_pivots_linear[lmcs_min_bin_idx+1]), the chroma scaling factor maybe derived based on the following equation.

InvScaleCoeff=OrgCW*(1<<11)/lmcs_pivots_linear[lmcs_min_bin_idx+1]  [Equation9]

In one example, if lmcs_cw_linear denotes a value mapped to P2 (i.e.,lmcs_pivots_linear[lmcs_max_bin_idx+1]), the chroma scaling factor maybe derived based on the following equation.

InvScaleCoeff=OrgCW*(lmcs_max_bin_idx−lmcs_max_bin_idx+1)*(1<<11)/lmcs_pivots_linear[lmcs_max_bin_idx+1]  [Equation10]

FIG. 11 illustrates one example of linear reshaping (or linearreshaping, linear mapping) according to an embodiment of the presentdocument. That is, in this embodiment, the use of a linear reshaper inLMCS is proposed. For example, this example in FIG. 11 may relate toforward linear reshaping (mapping).

In the existing example, the LMCS may use a piecewise linear mappingwith a fixed 16 pieces. This complicates the reshaper design sinceabrupt transition between pivot points may cause undesirable artefact.Also, for inverse luma mapping of reshaper, it requires to identify thepiecewise function index. The piecewise function index identificationprocess is an iteration process with many comparisons. Further more, forchroma residual scaling using the corresponding luma block average, italso requires such luma piecewise index identification process. This notonly has the complexity issue, it also causes the latency of chromaresidual scaling depending on reconstruction of whole luma block. Inorder to solve this problem, the use of the linear reshaper is proposedfor the LMCS.

Referring to FIG. 11, the linear reshaper may include two pivot pointsi.e., P1 and P2. P1 and P2 may represent input and mapped values, forexample P1 may be (min_input, 0) and P2 may be (max_input, max_mapped).Here, min_input represents the minimum input value, and max_inputrepresents the maximum input value. Any input value less than or equalto min_input are mapped to 0, any input value larger than max_input aremapped to max_mapped. Any input luma values within the min_input andmax_input are linearly mapped to other values. FIG. 11 shows an exampleof mapping. The pivot points P1, P2 may be determined at the encoder,and a linear fitting may be used to approximate the piecewise linearmapping.

There are many ways to signal the linear reshaper. In an example of amethod of signaling the linear reshaper, original luma range may bedivided by an equal number of bins. That is, the luma mapping betweenthe minimum and maximum bins may be equally distributed. For example,all bins may have same LMCS codeword (lmcsCW). To this end, minimum andmaximum bin indices may be signaled. Therefore, only one set ofreshape_model_bin_delta_abs_CW (or reshaper_model_delta_abs_CW,lmcs_delta_abs_CW) and reshaper_model_bin_delta_sign_CW_flag (orreshaper_model_delta_sign_CW_flag_sign_CW_flag) need to be signaled. Thefollowing table exemplarily shows the syntax of the reshaper modelaccording to this example.

TABLE 16 Descriptor reshaper_model ( ) { log2_reshape_model_num_bins_minus4 ue(v)   lmcs_min_bin_idx ue(v)  lmcs_max_bin_idx ue(v)   lmcs_delta_cw_prec_minus1 ue(v)  lmcs_delta_abs_CW u(v)   if ( lmcs_delta_abs_CW > 0 )  lmcs_sign_CW_flag u(1)

Referring to Table 16, the syntax element log2_reshape_model_num_bins_minus4 may be information on the number ofbins. Based on this information, the number of bins may be signaled toallow finer control of the minimum and maximum pivot points. In anotherexisting example, the encoder and/or decoder may derive (specify) the(fixed) number of bins without signaling, for example the number of binsmay be derived as 16 or 32. However, according to the examples of Table16, log 2_reshape_model_num_bins_minus4 plus 4 may represent a binarylogarithm of the number of bins.

For example, the LMCS codeword (lmcsCWALL) may be derived based on thefollowing equations.

lmcsDeltaCW=(1−2*lmcs_delta_sign_cw_flag)*lmcs_delta_abs_cw  [Equation11]

lmcsCW1=OrgCW+lmcsDeltaCW  [Equation 12]

lmcsCW[i]=lmcsCW1  [Equation 13]

In Equation 13, i may have a value ranging from lmcs_min_bin_idx tolmcs_max_bin_idx. That is, it can be seen from Equation 13 that all binshave the same LMCS codeword value (i.e., lmcsCW1).

In another example of a method of signaling the linear reshaper, an LMCScodeword (lmcsCWALL) may be derived based on the following equations.Also in this example, the syntax of Table 16 described above may beused.

lmcsDeltaCWAll=(1−2*lmcs_delta_sign_cw_flag)*lmcs_delta_abs_cw  [Equation14]

lmcsCWAll=lmcsDeltaCWAll+OrgCW*(lcms_max_bin_idx−lmcs_min_bin_idx+1)  [Equation15]

In this example and the above-mentioned equations, the LMCS codeword isexpressed as lmcsCWALL, but may be replaced with lmcsCW[i], in whichcase i may have any value from lmcs_min_bin_idx to lmcs_max_bin_idx.This is also the case in the following examples.

In another example of a method of signaling the linear reshaper, thepivot points P1, P2 of the linear reshaper model may be explicitlysignaled. The table below exemplifies the syntax of the reshaper modelaccording to this example.

TABLE 17 Descriptor reshaper_model ( ) {  lmcs_p1_x ue(v) lmcs_p2_x_delta ue(v)  lmcs_p2_y_delta ue(v)

Referring to Table 17, lmcs_p1_x may represent an input value of thefirst pivot point. For example, lmcs_p1_x may have a mapped value of 0.lmcs_p2_x_delta may represent a difference between the input value ofthe second pivot point and the maximum luma value (i.e.,(1<<bitdepthY)−1). lmcs_p2_y_delta may indicate a difference between themapped value of the second pivot point and the maximum luma value (i.e.,(1<<bitdepthY)−1).

The LMCS codeword lmcsCWALL may be derived based on the syntax elementsdescribed above. For example, lmcsCWALL may be derived based on thefollowing equation.

lmcsCWAll=lmcs_p2_y=(1<<bitdepthY)−1−lmcs_p2_y_delta  [Equation 16]

Inverse scaling for luma samples may be performed based on theabove-described examples of the linear reshaper. In one example, onlyone inverse scaling factor may be needed for inverse scaling for lumasamples. The inverse luma mapping scaling factor may be derived based onat least one of the following equations.

InvScaleCoeff=OrgCW/lmcsCW1  [Equation 17]

InvScaleCoeff=OrgCW*(lmcs_max_bin_idx−lmcs_max_bin_idx+1)/lmcsCWAll  [Equation18]

InvScaleCoeff=(lmcs_p2_x−lmcs_p1_x)/lmcsCWAll  [Equation 19]

Based on the above-described examples of the linear reshaper, thepiecewise index identification process may be omitted. That is, in thepresent examples, since there is only one piece having valid reshapedluma pixels, the piecewise index identification process used for inverseluma mapping and chroma residual scaling can be removed. Accordingly,the complexity of inverse luma mapping may be reduced. In addition,latency problems caused by depending on luma piecewise indexidentification during chroma residual scaling can be eliminated.

According to the embodiment of the use of the linear reshaper describedabove, the following advantages may be provided for LMCS: i) It ispossible to simplify the encoder reshaper design, preventing possibleartifact by abrupt changes between the piecewise linear pieces ii) Thedecoder inverse mapping process, in which the piecewise indexidentification process can be removed, can be simplified by eliminatingthe piecewise index identification process iii) By removing thepiecewise index identification process, it is possible to remove thelatency problem in the chroma residual scaling caused by depending onthe corresponding luma blocks iv) It is possible to reduce overhead ofsignaling, and make frequent update of reshaper more feasible v) Formany places that used to require of a loop of 16 pieces, the loop can beeliminated. For example, to derive InvScaleCoeff[i], the number ofdivision operations by lmcsCW[i] can be reduced to 1.

The following drawings are created to explain specific examples of thepresent specification. Since the names of specific devices described inthe drawings or the names of specific signals/messages/fields arepresented by way of example, the technical features of the presentspecification are not limited to the specific names used in thefollowing drawings.

FIG. 12 and FIG. 13 schematically show an example of a video/imageencoding method and related components according to embodiment(s) of thepresent document. The method disclosed in FIG. 12 may be performed bythe encoding apparatus disclosed in FIG. 2. Specifically, for example,S1200 may be performed by the residual processor 230, the adder 250and/or the filter 260 of the encoding apparatus, S1210 may be performedby the residual processor 230 of the encoding apparatus, S1220 may beperformed by the adder 250 of the encoding apparatus, S1230 may beperformed by the filter 260 of the encoding apparatus, S1240 may beperformed by the residual processor 230, the adder 250 and/or the filter260 of the encoding apparatus, and S1250 may be performed by the entropyencoder 240 of the encoding apparatus. The method disclosed in FIG. 12may include the embodiments described above in the present document.

Referring to FIG. 12, the encoding apparatus may derive at least oneLMCS codeword (S1200). In one example, the at least one LMCS codewordmay be generated based on the original codeword based on the luma bitdepth. For example, the number of at least one LMCS codeword may beequal to the number of bins used for the LMCS. The encoding apparatusmay derive one LMCS codeword for one bin based on the linear mapping.

The encoding apparatus may generate residual information (S1210). Theencoding apparatus may derive residual samples for the current block,and the residual samples for the current block may be derived based onoriginal samples and prediction samples of the current block.Specifically, the encoding apparatus may derive the prediction samplesof the current block based on the prediction mode. In this case, variousprediction methods disclosed in the present document, such as interprediction or intra prediction, may be applied. Residual samples may bederived based on the prediction samples and the original samples. Theencoding apparatus may derive transform coefficients based on atransform procedure for the residual samples. For example, the transformprocess may include at least one of DCT, DST, GBT, or CNT. The encodingapparatus may derive quantized transform coefficients based on thequantization process for the transform coefficients. The quantizedtransform coefficients may have a one-dimensional vector form based onthe coefficient scan order. The encoding apparatus may generate residualinformation specifying the quantized transform coefficients. Theresidual information may be generated through various encoding methodssuch as exponential Golomb, CAVLC, CABAC, and the like.

The encoding apparatus may generate reconstructed luma samples (S1220).The encoding apparatus may generate reconstructed luma samples based onthe at least one LMCS codeword and the residual information. Thereconstructed luma samples may be generated based on residual samplesbased on residual information and prediction samples. For example, thereconstructed luma samples may be generated based on the summationbetween prediction samples mapped according to forward mapping andresidual samples.

The encoding apparatus may generate modified reconstructed luma samples(S1230). The encoding apparatus may generate modified reconstructed lumasamples based on the reconstructed luma samples and the at least oneLMCS codeword. For example, the modified reconstructed luma samples maybe generated according to the features described in Tables 7 to 10described above.

The encoding apparatus may derive LMCS related information (S1240). Theencoding apparatus may generate LMCS related information for thereconstructed samples. The encoding apparatus may derive LMCS relatedparameters that can be applied for filtering the reconstructed samples,and may generate LMCS related information based on the LMCS relatedparameters. For example, the LMCS related information may includeinformation on the luma mapping (i.e., forward mapping, inverse mapping,linear mapping), information on chroma residual scaling, and/or indices(i.e., a maximum bin index, a minimum bin index) related to LMCS (orreshaping, reshaper).

The encoding apparatus may encode the video/image information (S1250).The image information may include LMCS related information and/orresidual information. For example, the LMCS related information mayinclude information on the linear mapping. In one example, at least oneLMCS codeword may be derived based on information on the linear mapping.The encoded video/image information may be output in the form of abitstream. The bitstream may be transmitted to the decoding devicethrough a network or a storage medium.

The image/video information may include various information according toan embodiment of the present document. For example, the image/videoinformation may include information disclosed in at least one of Tables1, 3, 4, 7, 14, 15, 16 and/or 17 described above.

In one embodiment, an LMCS delta codeword may be derived based on atleast one LMCS codeword and an original codeword (OrgCW). In oneexample, the information on the linear mapping may include informationon the LMCS delta codeword.

In one embodiment, the at least one LMCS codeword may be derived basedon the summation of the LMCS delta codeword and OrgCW, for example,OrgCW may be (1<<BitDepthY)/16, where BitDepthY may be a luma bit depth.This embodiment may be based on Equation 12.

In one embodiment, the at least one LMCS codeword may be derived basedon the summation of the LMCS delta codeword andOrgCW*(lmcs_max_bin_idx−lmcs_min_bin_idx+1), and lmcs_max_bin_idx andlmcs_min_bin_idx may be a maximum bin index and a minimum bin index,respectively, and OrgCW may be (1<<BitDepthY)/16. This embodiment may bebased on Equation 15.

In one embodiment, the information on the linear mapping may includeinformation on two pivot points. For example, the two pivot points mayinclude a first pivot point (P1 in FIG. 11) and a second pivot point (P2in FIG. 11). The x value and y value of the first pivot point are aminimum input value (minInput) and a minimum mapping value (i.e., 0),respectively, and the x value and the y value of the second pivot pointare a maximum input value (maxInput) and the maximum mapping value(maxMapped), respectively. Input values between the minimum input valueand the maximum input value may be linearly mapped.

In one embodiment, the information on the linear mapping may include thex delta value (lmcs_p1_x_delta) of the first pivot point and the y deltavalue (lmcs_p2_y_delta) of the second pivot point, and the maximum inputvalue (maxInput) may be derived based on the x delta value(lmcs_p2_x_delta) of the second pivot point, and the maximum mappingvalue (i.e., lmcsCWALL, lmcs_p2_y in Equation 16) may be derived basedon the y delta value (lmcs_p2_y_delta) of the second pivot point.

In one embodiment, reconstructed chroma samples may be derived based onthe LMCS related information, and the reconstructed chroma samples maybe derived based on a single chroma residual scaling factor. Through thederivation of reconstructed chroma samples based on the single chromaresidual scaling factor, scaling in a chroma block may not depend on aluma block value and piecewise index identification may not be required.Therefore, the chroma residual scaling process in the LMCS may beperformed without a latency issue.

In one embodiment, the single chroma residual scaling factor may bederived based on an average of inverse mapping gradients (slopes) forderiving the modified reconstructed luma samples. For example, thisembodiment may be based on Table 12 described above.

In one embodiment, a gradient reciprocal (slope reciprocal) of thelinear mapping may be used as the single chroma residual scaling factorbased on the information on the linear mapping. For example, thisembodiment may be based on Table 13 described above.

In one embodiment, the LMCS related information may include informationon the parameters for the single chroma residual scaling factor, and thesingle chroma residual scaling factor may be derived based on theparameters for the single chroma residual scaling factor. For example,the parameters for the single chroma residual scaling factor may includelmcs_cw_linear included in Table 14 or 15 described above, a valuemapped to arbitrary pivot points of linear mapping, a y value atarbitrary linear pivot points, lmcs_pivots_linear [lmcs_max_bin_idx+1]and lmcs_pivots_linear[lmcs_min_bin_idx+1].

In one embodiment, the at least one LMCS codeword may be a multiple oftwo.

In one embodiment, when the luma bit depth (BitDepthY) of thereconstructed luma samples is higher than 10, the at least one LMCScodeword may be a multiple of 1<<(BitDepthY−10).

In one embodiment, the at least one LMCS codeword may be in the rangefrom (OrgCW>>1) to (OrgCW<<1)−1.

FIG. 14 and FIG. 15 schematically show an example of an image/videodecoding method and related components according to an embodiment of thepresent document. The method disclosed in FIG. 14 may be performed bythe decoding apparatus illustrated in FIG. 3. Specifically, for example,S1400 of FIG. 14 may be performed by the entropy decoder 310 of thedecoding apparatus, S1410 may be performed by the residual processor320, the adder 340, and/or the filter 350 of the decoding apparatus,S1420 may be performed by the adder 340 of the decoding apparatus, andS1430 may be performed by the filter 350 of the decoding apparatus. Themethod disclosed in FIG. 14 may include the embodiments described abovein the present document.

Referring to FIG. 14, the decoding apparatus may receive/obtainvideo/image information (S1400). The video/image information may includeLMCS related information. For example, the LMCS related information mayinclude information on the luma mapping (i.e., forward mapping, inversemapping, linear mapping), information on chroma residual scaling, and/orindices (i.e., a maximum bin index, a minimum bin index) related to LMCS(or reshaping, reshaper). The decoding apparatus may receive/obtain theimage/video information through a bitstream.

The image/video information may include various information according toan embodiment of the present document. For example, the image/videoinformation may include information disclosed in at least one of Tables1, 3, 4, 7, 14, 15, 16 and/or 17 described above.

The decoding apparatus may derive at least one LMCS codeword (S1410).The decoding apparatus may derive at least one LMCS codeword for theLMCS process based on the LMCS related information. In one example, theat least one LMCS codeword may be generated based on an originalcodeword based on a luma bit depth. The at least one LMCS codeword maybe derived based on the features described together with Tables 7 and 8described above. For example, the number of the at least one LMCScodeword may be the same as the number of bins used for the LMCS. Thedecoding apparatus may derive one LMCS codeword for one bin based on thelinear mapping.

The decoding apparatus may generate reconstructed luma samples (S1420).The decoding apparatus may generate reconstructed luma samples based onthe at least one LMCS codeword. Specifically, the decoding apparatus maygenerate prediction samples by performing prediction on the currentblock, and may derive residual samples based on residual informationincluded in the image information. Reconstructed luma samples may begenerated based on residual samples and prediction samples. For example,the reconstructed luma samples may be generated based on the summationbetween predicted samples and residual samples mapped according toforward mapping using the at least one LMCS codeword.

The decoding apparatus may generate modified reconstructed luma samples(S1430). The decoding apparatus may generate modified reconstructed lumasamples based on the reconstructed luma samples and the at least oneLMCS codeword. For example, the modified reconstructed luma samples maybe generated according to the features described in Tables 7 to 10described above.

In one embodiment, the LMCS related information may include informationon linear mapping. In an example, the at least one LMCS codeword may bederived based on the information on the linear mapping.

In one embodiment, an LMCS delta codeword may be derived based on theinformation on the linear mapping.

In one embodiment, the at least one LMCS codeword may be derived basedon the summation of the LMCS delta codeword and OrgCW, for example,OrgCW may be (1<<BitDepthY)/16, where BitDepthY may be a luma bit depth.This embodiment may be based on Equation 12.

In one embodiment, the at least one LMCS codeword may be derived basedon the summation of the LMCS delta codeword andOrgCW*(lmcs_max_bin_idx−lmcs_min_bin_idx+1), and lmcs_max_bin_idx andlmcs_min_bin_idx may be a maximum bin index and a minimum bin index,respectively, and OrgCW may be (1<<BitDepthY)/16. This embodiment may bebased on Equation 15.

In one embodiment, information on two pivot points may be derived basedon the information on the linear mapping. For example, the two pivotpoints may include a first pivot point (P1 in FIG. 11) and a secondpivot point (P2 in FIG. 11). The x value and y value of the first pivotpoint are a minimum input value (minInput) and a minimum mapping value(i.e., 0), respectively, and the x value and the y value of the secondpivot point are a maximum input value (maxInput) and the maximum mappingvalue (maxMapped), respectively. Input values between the minimum inputvalue and the maximum input value may be linearly mapped.

In one embodiment, the x delta value (lmcs_p1_x_delta) of the firstpivot point and the y delta value (lmcs_p2_y_delta) of the second pivotpoint may signaled based on the information on the linear mapping, andthe maximum input value (maxInput) may be derived based on the x deltavalue (lmcs_p2_x_delta) of the second pivot point, and the maximummapping value (i.e., lmcsCWALL, lmcs_p2_y in Equation 16) may be derivedbased on the y delta value (lmcs_p2_y_delta) of the second pivot point.

In one embodiment, reconstructed chroma samples may be derived based onthe LMCS related information, and the reconstructed chroma samples maybe derived based on a single chroma residual scaling factor. Through thederivation of reconstructed chroma samples based on the single chromaresidual scaling factor, scaling in a chroma block may not depend on aluma block value and piecewise index identification may not be required.Therefore, the chroma residual scaling process in the LMCS may beperformed without a latency issue.

In one embodiment, the single chroma residual scaling factor may bederived based on an average of inverse mapping gradients (slopes) forderiving the modified reconstructed luma samples. For example, thisembodiment may be based on Table 12 described above.

In one embodiment, a gradient reciprocal (slope reciprocal) of thelinear mapping may be used as the single chroma residual scaling factorbased on signaling of the information on the linear mapping. Forexample, this embodiment may be based on Table 13 described above.

In one embodiment, the parameters for the single chroma residual scalingfactor may be signaled based on the LMCS related information, and thesingle chroma residual scaling factor may be derived based on theparameters for the single chroma residual scaling factor. For example,the parameters for the single chroma residual scaling factor may includelmcs_cw_linear included in Table 14 or 15 described above, a valuemapped to arbitrary pivot points of linear mapping, a y value atarbitrary linear pivot points, lmcs_pivots_linear [lmcs_max_bin_idx+1]and lmcs_pivots_linear[lmcs_min_bin_idx+1].

In one embodiment, the at least one LMCS codeword may be a multiple oftwo.

In one embodiment, when the luma bit depth (BitDepthY) of thereconstructed luma samples is higher than 10, the at least one LMCScodeword may be a multiple of 1<<(BitDepthY−10).

In one embodiment, the at least one LMCS codeword may be in the rangefrom (OrgCW>>1) to (OrgCW<<1)−1.

In the above-described embodiment, the methods are described based onthe flowchart having a series of steps or blocks. The present disclosureis not limited to the order of the above steps or blocks. Some steps orblocks may occur simultaneously or in a different order from other stepsor blocks as described above. Further, those skilled in the art willunderstand that the steps shown in the above flowchart are notexclusive, that further steps may be included, or that one or more stepsin the flowchart may be deleted without affecting the scope of thepresent disclosure.

The method according to the above-described embodiments of the presentdocument may be implemented in software form, and the encoding deviceand/or decoding device according to the present document is, forexample, may be included in the device that performs the imageprocessing of a TV, a computer, a smart phone, a set-top box, a displaydevice, etc.

When the embodiments in the present document are implemented insoftware, the above-described method may be implemented as a module(process, function, etc.) that performs the above-described function. Amodule may be stored in a memory and executed by a processor. The memorymay be internal or external to the processor, and may be coupled to theprocessor by various well-known means. The processor may include anapplication-specific integrated circuit (ASIC), other chipsets, logiccircuits, and/or data processing devices. Memory may include read-onlymemory (ROM), random access memory (RAM), flash memory, memory cards,storage media, and/or other storage devices. That is, the embodimentsdescribed in the present document may be implemented and performed on aprocessor, a microprocessor, a controller, or a chip. For example, thefunctional units shown in each figure may be implemented and performedon a computer, a processor, a microprocessor, a controller, or a chip.In this case, information on instructions or an algorithm forimplementation may be stored in a digital storage medium.

In addition, the decoding apparatus and the encoding apparatus to whichthe present disclosure is applied may be included in a multimediabroadcasting transmission/reception apparatus, a mobile communicationterminal, a home cinema video apparatus, a digital cinema videoapparatus, a surveillance camera, a video chatting apparatus, areal-time communication apparatus such as video communication, a mobilestreaming apparatus, a storage medium, a camcorder, a VoD serviceproviding apparatus, an Over the top (OTT) video apparatus, an Internetstreaming service providing apparatus, a three-dimensional (3D) videoapparatus, a teleconference video apparatus, a transportation userequipment (i.e., vehicle user equipment, an airplane user equipment, aship user equipment, etc.) and a medical video apparatus and may be usedto process video signals and data signals. For example, the Over the top(OTT) video apparatus may include a game console, a blue-ray player, aninternet access TV, a home theater system, a smart phone, a tablet PC, aDigital Video Recorder (DVR), and the like.

Furthermore, the processing method to which the present document isapplied may be produced in the form of a program that is to be executedby a computer and may be stored in a computer-readable recording medium.Multimedia data having a data structure according to the presentdisclosure may also be stored in computer-readable recording media. Thecomputer-readable recording media include all types of storage devicesin which data readable by a computer system is stored. Thecomputer-readable recording media may include a BD, a Universal SerialBus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, a magnetic tape, afloppy disk, and an optical data storage device, for example.Furthermore, the computer-readable recording media includes mediaimplemented in the form of carrier waves (i.e., transmission through theInternet). In addition, a bitstream generated by the encoding method maybe stored in a computer-readable recording medium or may be transmittedover wired/wireless communication networks.

In addition, the embodiments of the present document may be implementedwith a computer program product according to program codes, and theprogram codes may be performed in a computer by the embodiments of thepresent document. The program codes may be stored on a carrier which isreadable by a computer.

FIG. 16 shows an example of a content streaming system to whichembodiments disclosed in the present document may be applied.

Referring to FIG. 16, the content streaming system to which theembodiment(s) of the present document is applied may largely include anencoding server, a streaming server, a web server, a media storage, auser device, and a multimedia input device.

The encoding server compresses content input from multimedia inputdevices such as a smartphone, a camera, a camcorder, etc. Into digitaldata to generate a bitstream and transmit the bitstream to the streamingserver. As another example, when the multimedia input devices such assmartphones, cameras, camcorders, etc. directly generate a bitstream,the encoding server may be omitted.

The bitstream may be generated by an encoding method or a bitstreamgenerating method to which the embodiment(s) of the present disclosureis applied, and the streaming server may temporarily store the bitstreamin the process of transmitting or receiving the bitstream.

The streaming server transmits the multimedia data to the user devicebased on a user's request through the web server, and the web serverserves as a medium for informing the user of a service. When the userrequests a desired service from the web server, the web server deliversit to a streaming server, and the streaming server transmits multimediadata to the user. In this case, the content streaming system may includea separate control server. In this case, the control server serves tocontrol a command/response between devices in the content streamingsystem.

The streaming server may receive content from a media storage and/or anencoding server. For example, when the content is received from theencoding server, the content may be received in real time. In this case,in order to provide a smooth streaming service, the streaming server maystore the bitstream for a predetermined time.

Examples of the user device may include a mobile phone, a smartphone, alaptop computer, a digital broadcasting terminal, a personal digitalassistant (PDA), a portable multimedia player (PMP), navigation, a slatePC, tablet PCs, ultrabooks, wearable devices (ex. Smartwatches, smartglasses, head mounted displays), digital TVs, desktops computer, digitalsignage, and the like. Each server in the content streaming system maybe operated as a distributed server, in which case data received fromeach server may be distributed.

Each server in the content streaming system may be operated as adistributed server, and in this case, data received from each server maybe distributed and processed.

The claims described herein may be combined in various ways. Forexample, the technical features of the method claims of the presentdocument may be combined and implemented as an apparatus, and thetechnical features of the apparatus claims of the present document maybe combined and implemented as a method. In addition, the technicalfeatures of the method claim of the present document and the technicalfeatures of the apparatus claim may be combined to be implemented as anapparatus, and the technical features of the method claim of the presentdocument and the technical features of the apparatus claim may becombined and implemented as a method.

What is claimed is:
 1. An image decoding method performed by a decodingapparatus, the method comprising: obtaining image information includingluma mapping with chroma scaling (LMCS) related information from abitstream; deriving at least one LMCS codeword for an LMCS process basedon the LMCS related information; generating reconstructed luma samplesbased on the at least one LMCS codeword; and generating modifiedreconstructed luma samples based on the reconstructed luma samples andthe at least one LMCS codeword, wherein the LMCS related informationincludes information on linear mapping, wherein the at least one LMCScodeword is derived based on signaling of the information on the linearmapping.
 2. The method of claim 1, wherein an LMCS delta codeword isderived based on the information on the linear mapping, wherein the atleast one LMCS codeword is derived based on the sum of the LMCS deltacodeword and OrgCW, wherein OrgCW is equal to (1<<BitDepthY)/16, andwherein BitDepthY represents a luma bit depth.
 3. The method of claim 1,wherein an LMCS delta codeword is derived based on the information onthe linear mapping, wherein the at least one LMCS codeword is derivedbased on the sum of the LMCS delta codeword andOrgCW*(lmcs_max_bin_idx-lmcs_min_bin_idx+1), wherein lmcs_max_bin_idxand lmcs_min_bin_idx are a maximum bin index and a minimum bin index,respectively, wherein OrgCW is (1<<BitDepthY)/16, and wherein BitDepthYrepresents a luma bit depth.
 4. The method of claim 1, whereininformation on two pivot points is derived based on the information onthe linear mapping, wherein the two pivot points include a first pivotpoint and a second pivot point, wherein an x value and a y value of thefirst pivot point are a minimum input value and a minimum mapping value,respectively, wherein an x value and a y value of the second pivot pointare a maximum input value and a maximum mapping value, respectively,wherein input values between the minimum input value and the maximuminput value are linearly mapped.
 5. The method of claim 4, wherein an xdelta value of the first pivot point and a y delta value of the secondpivot point are signaled based on the information on the linear mapping,wherein the maximum input value is derived based on the x delta value ofthe second pivot point, and wherein the maximum mapping value is derivedbased on the y delta value of the second pivot point.
 6. The method ofclaim 5, wherein the maximum mapping value is derived based on thefollowing equation,lmcsCWAll=lmcs_p2_y=(1<<bitdepthY)−1−lmcs_p2_y_delta, herein, lmcsCWALLand lmcs_p2_y represent the maximum mapping value, bitdepthY representsa luma bit depth, and lmcs_p2_y_delta represents the y delta value ofthe second pivot point.
 7. The method of claim 1, wherein the methodfurther comprises deriving reconstructed chroma samples based on theLMCS related information, and wherein the reconstructed chroma samplesare derived based on a single chroma residual scaling factor.
 8. Themethod of claim 7, wherein the single chroma residual scaling factor isderived based on an average of inverse mapping gradients for derivingthe modified reconstructed luma samples.
 9. The method of claim 7,wherein based on signaling of the information on the linear mapping, agradient reciprocal of the linear mapping is used as the single chromaresidual scaling factor.
 10. The method of claim 7, wherein parametersfor the single chroma residual scaling factor are signaled based on theLMCS related information, and wherein the single chroma residual scalingfactor is derived based on the parameters for the single chroma residualscaling factor.
 11. The method of claim 1, wherein the at least one LMCScodeword is a multiple of two.
 12. The method of claim 1, when a lumabit depth of the reconstructed luma samples is higher than 10, the atleast one LMCS codeword is a multiple of 1<<(BitDepthY−10), and herein,BitDepthY represents the luma bit depth.
 13. The method of claim 1,wherein the at least one LMCS codeword is in a range from (OrgCW>>1) to(OrgCW<<1)−1.
 14. An image encoding method performed by an encodingapparatus, the method comprising: deriving at least one luma mappingwith chroma scaling (LMCS) codeword; generating residual informationbased on prediction for a current block; generating reconstructed lumasamples based on the at least one LMCS codeword and the residualinformation; generating modified reconstructed luma samples based on theat least one LMCS codeword and the reconstructed luma samples;generating LMCS related information based on the at least one LMCScodeword, the reconstructed luma samples, and the modified reconstructedluma samples; and encoding image information including the residualinformation and the LMCS related information, wherein the LMCS relatedinformation includes information on linear mapping, wherein the at leastone LMCS codeword is derived based on the information on the linearmapping.
 15. A computer-readable digital storage medium, storing encodedinformation causing a decoding apparatus to perform an image decodingmethod, the method comprising: obtaining image information includingluma mapping with chroma scaling (LMCS) related information from abitstream; deriving at least one LMCS codeword for an LMCS process basedon the LMCS related information; generating reconstructed luma samplesbased on the at least one LMCS codeword; and generating modifiedreconstructed luma samples based on the reconstructed luma samples andthe at least one LMCS codeword, wherein the LMCS related informationincludes information on linear mapping, wherein the at least one LMCScodeword is derived based on signaling of the information on the linearmapping.